01.03 · Lecture
Little's Law
State L = lambda x W and apply it to a queue, knowing it holds for any arrival pattern and any service order.
Little's Law is L = lambda x W: average in-flight work equals throughput times average time in the system. It applies to stable queues and services regardless of burstiness or scheduling policy. The useful trick is choosing the right boundary, matching units, and using admitted or completed work rather than offered load.
What this lesson answers
- how do I apply Little's Law to queues
- does Little's Law require FIFO service order
- which throughput rate should Little's Law use
Notes
Little's Law — Little's Law exists to connect throughput, waiting time, and in-flight work in any stable system; without it, queues are often misdiagnosed because latency changes are mistaken for capacity changes.
Key Concepts: - The law is , where is average number of items in the system, is average arrival or completion rate, and is average time an item spends in the system. - If a service completes requests/second and average request time is seconds, then average concurrency is requests.
References
Common questions
- What does Little's Law mean in production systems?
- It gives a consistency check between concurrency, throughput, and latency. If you know the completion rate and average time spent inside a chosen boundary, their product is the average work inside that boundary. That can mean queued jobs, active requests, Lambda concurrency, database connections, or total in-flight work.
- Does Little's Law still work with bursty traffic?
- Yes, provided the system is stable over the measurement interval. The arrival pattern does not need to be smooth or Poisson. Bursts, batches, and uneven traffic still satisfy the relationship when arrivals and departures balance over a long enough window and you measure the same boundary for work and time.
- Why does the system boundary matter for Little's Law?
- The average count and average time must describe the same part of the system. Queue depth belongs with queue waiting time. End-to-end latency belongs with all in-flight work across waiting, service, network effects, and retries. Mixing a queue-only metric with a wider latency metric produces a misleading result.
Short definition: what is Little's Law?