10.01 · Lecture
The Curve, Collected
Recognise the 1/(1-utilisation) curve as the shape underneath saturation, retries and queue growth alike.
Shared services do not degrade linearly near saturation: the waiting term follows the same 1/(1-utilisation) shape behind queues, retries and latency blow-ups. Once utilisation is high, a small increase in offered load can create a much larger increase in backlog, tail latency and retry traffic.
What this lesson answers
- why does latency spike near high utilisation
- how do retries make overloaded services worse
- what does Little’s Law say about backlog
Notes
The 1/(1-utilisation) Curve — The curve exists because shared systems slow down nonlinearly as utilisation approaches 1; without recognising it, teams misread small load increases near saturation as harmless and trigger runaway latency, queues, retries, and outages.
Key Concepts: - In an queue, mean response time is where is service time and ; at , latency is . - Queue length follows in an system; moving from to raises expected queued work from …
References
Common questions
- Why is high utilisation dangerous even before a system is fully busy?
- Because shared capacity creates waiting, not just service time. As utilisation gets close to full, each arriving request is more likely to find work already ahead of it. The queueing delay then grows much faster than the load increase, so a system can look only slightly busier while response times become dramatically worse.
- How can retries turn a slow dependency into an outage?
- Retries add more work to the same dependency that is already struggling. If callers retry aggressively, the effective arrival rate rises just when the service has the least spare capacity. That extra traffic pushes the system further up the nonlinear part of the curve, increasing queues, timeouts and still more retries.
- What does Little’s Law add to incident analysis?
- Little’s Law connects throughput, latency and in-flight work. If request rate stays high while latency rises, the amount of work inside the system must also rise. That makes backlog visible as a consequence of measured traffic and waiting time, rather than as a vague symptom of a slow service.
Short definition: what is Curve, Collected?