10.04 · Lecture
Backpressure
Shed load deliberately, and read latency = A + B x concurrency as the same curve from Module 1.
Backpressure keeps overload explicit by slowing, rejecting, or dropping work before queues and retries turn latency into failure. The same latency curve shows up as base service time plus contention cost multiplied by concurrency, so bounding queues and concurrency is a reliability control, not just a performance tweak.
What this lesson answers
- how does backpressure prevent cascading failures
- when should a service shed load deliberately
- why do retries make overloaded systems worse
Notes
Backpressure — Backpressure exists to stop faster producers from overwhelming slower consumers; without it, queues grow until latency explodes, memory fills, retries amplify load, and the system fails instead of degrading deliberately.
Key Concepts: - Latency under contention follows the same curve as Module 1: , where is base service time, is added delay per concurrent request, and is concurrency. - Little’s Law connects queue size, throughput, and latency: , so at requests/s and ms wait, the queue holds about…
References
Common questions
- What is backpressure in a production system?
- Backpressure is a control mechanism that stops producers from sending more work than consumers can process. It may appear as bounded queues, concurrency limits, flow-control windows, circuit breakers, or explicit rejections. The goal is to make overload visible and contained instead of letting memory, latency, and retries grow without limit.
- Why is an unbounded queue dangerous during overload?
- An unbounded queue hides the overload at first, then converts it into rising latency and memory pressure. Requests keep being accepted even though they cannot be served promptly. By the time CPU or heap alarms fire, callers may already be timing out, retrying, and adding still more work to the system.
- How should a service shed load safely?
- A service should shed load at clear boundaries using limits it can enforce, such as queue depth, active worker count, pending requests, or latency over repeated observation windows. When limits are reached, it should reject, return overload responses, or drop lower-priority work rather than accepting requests it cannot complete usefully.
Short definition: what is Backpressure?