Why is exponential backoff alone not enough?

Because it changes the spacing of retries for each client, not the relationship between clients. If many callers fail together, they can all sleep for the same sequence of delays and then retry together. The peaks move farther apart, but the herd remains synchronised unless randomness is added.

What does jitter actually do?

Jitter randomises the retry delay, usually within a cap derived from the backoff schedule. Instead of every caller sleeping for the same computed duration, each caller picks a different delay. That decorrelates arrivals, turning a sharp burst into a flatter stream the recovering service has a better chance of serving.

Should every failed request be retried?

No. It depends on whether the operation is idempotent, whether the error is likely transient, and whether the caller still has time to get a useful answer. Retrying overload errors without a budget can make the outage worse. Retrying permanent validation failures or unsafe writes can be both wasteful and incorrect.

What is retry amplification?

Retry amplification happens when retries exist at several layers or across fan-out calls. A single inbound request may call multiple downstream services, and each of those clients may retry. The apparent traffic at the edge can therefore become much more work inside the system, especially during a shared dependency failure.

Retry Storms

A retry storm is a failure mode where many clients repeat requests to the same unhealthy dependency at nearly the same time, adding load precisely when it has least capacity. If retries are synchronised across clients or layers, a short outage can become sustained overload as recovery is repeatedly crushed.

Retrying is meant to hide transient faults, but it becomes dangerous when a large population observes the same timeout or error together. Each caller believes it is being conservative, yet the dependency sees a second wave of work on top of the original traffic. If the service was slow because it was saturated, those extra attempts make queues longer, increase timeouts, and cause still more callers to retry.

Backoff reduces how often each individual client retries, but it does not by itself spread clients apart. If all clients start at the same failure boundary, exponential delays still put them on the same later boundaries. Jitter fixes the coordination problem by choosing a random sleep within a capped range, so retries smear across time instead of arriving as a herd.

The trade-off is that jitter makes individual request latency less predictable. Some callers will wait longer than they would under a fixed schedule, and some strategies allow very short sleeps unless deliberately avoided. The right policy depends on the dependency, the operation’s cost, user-visible deadlines, and whether retrying is safe. Backoff protects frequency; jitter protects concurrency; retry budgets limit total damage.

Engineers meet retry storms in HTTP clients, SDKs, service meshes, queues, workers, mobile apps, and database drivers. The common trap is letting every layer retry independently, so one user action turns into many downstream attempts. Look for retry policies on HTTP 500 responses, fixed sleep loops, mesh-level retries, and client libraries that lack randomised truncated exponential backoff.

Go deeper

DeploymentRetry StormsExplain why backoff alone does not stop a herd and why jitter is the part that decorrelates it.Open the lesson

Common questions

Why is exponential backoff alone not enough?: Because it changes the spacing of retries for each client, not the relationship between clients. If many callers fail together, they can all sleep for the same sequence of delays and then retry together. The peaks move farther apart, but the herd remains synchronised unless randomness is added.
What does jitter actually do?: Jitter randomises the retry delay, usually within a cap derived from the backoff schedule. Instead of every caller sleeping for the same computed duration, each caller picks a different delay. That decorrelates arrivals, turning a sharp burst into a flatter stream the recovering service has a better chance of serving.
Should every failed request be retried?: No. It depends on whether the operation is idempotent, whether the error is likely transient, and whether the caller still has time to get a useful answer. Retrying overload errors without a budget can make the outage worse. Retrying permanent validation failures or unsafe writes can be both wasteful and incorrect.
What is retry amplification?: Retry amplification happens when retries exist at several layers or across fan-out calls. A single inbound request may call multiple downstream services, and each of those clients may retry. The apparent traffic at the edge can therefore become much more work inside the system, especially during a shared dependency failure.

Every term