Latency budget

1. Stage 1500.0 us at p50L1 cache referenceBranch mispredictL2 cache referenceMutex lock/unlockMain memory referenceCompress 1KB with ZippySend 1KB over 1GbpsSSD random readRead 1MB sequentially from memoryRound trip within a datacentreRead 1MB sequentially from SSDDisk seekRead 1MB sequentially from diskRound trip CA to Netherlands

   Calls1

   In parallel (wait for the slowest)
   Remove stage 1
2. Stage 21.9 ms at p50L1 cache referenceBranch mispredictL2 cache referenceMutex lock/unlockMain memory referenceCompress 1KB with ZippySend 1KB over 1GbpsSSD random readRead 1MB sequentially from memoryRound trip within a datacentreRead 1MB sequentially from SSDDisk seekRead 1MB sequentially from diskRound trip CA to Netherlands

   Calls50

   In parallel (wait for the slowest)
   Remove stage 2
3. Stage 377.5 us at p50L1 cache referenceBranch mispredictL2 cache referenceMutex lock/unlockMain memory referenceCompress 1KB with ZippySend 1KB over 1GbpsSSD random readRead 1MB sequentially from memoryRound trip within a datacentreRead 1MB sequentially from SSDDisk seekRead 1MB sequentially from diskRound trip CA to Netherlands

   Calls4

   In parallel (wait for the slowest)
   Remove stage 3
4. Stage 410.0 us at p50L1 cache referenceBranch mispredictL2 cache referenceMutex lock/unlockMain memory referenceCompress 1KB with ZippySend 1KB over 1GbpsSSD random readRead 1MB sequentially from memoryRound trip within a datacentreRead 1MB sequentially from SSDDisk seekRead 1MB sequentially from diskRound trip CA to Netherlands

   Calls1

   In parallel (wait for the slowest)
   Remove stage 4

This path costs 2.4 ms at the median and 3.6 ms at the 99th percentile, across 56 calls, of which 43.0 percent of requests touch at least one slow one.

43.0% of requests touch at least one p99-slow call. That is 1 - 0.99^56, and it is the whole argument for caring about the tail. A dependency that is slow one time in a hundred is not a rare event once a request makes 56 of them. At 100 calls it is 63.4% - the tail stops being an edge case and becomes the common case.

The third tile is the mistake this page exists to price. Adding per-stage p99s assumes every stage has its bad day on the same request, which is 22% pessimistic here. Summing the means and the variances and reading the percentile off the result is the right arithmetic, and it is what the second tile does.

A parallel stage costs the SLOWEST of its calls, not their sum, and the arithmetic for that is exact rather than simulated: the p-th percentile of the maximum of N independent draws is the p^(1/N)-th percentile of one draw. So the median of a 50-way fan-out is already the p98.6 of a single call. Fanning out does not remove the latency, it converts your median into somebody else's tail.

Real: the distribution fitting, the serial sum over means and variances, the exact parallel-maximum percentile, and the tail-amplification figure are all computed on this page from the constants shown. The inverse-normal function is a numerical method written out in the file, not a lookup table.

Standing in: the constants are published order-of-magnitude figures in the "latency numbers every programmer should know" lineage, not measurements of your hardware, and the per-operation p99s are illustrative - a real one comes from your own histogram and it is the input that moves the answer most. The log-normal shape is an assumption: real service latency is often bimodal, with a GC pause or a connection-pool wait forming a second hump this cannot draw. And the stages are treated as independent when in reality they queue behind the same CPUs and the same pool, so their slow moments arrive together.

Both of those last two push the same way. The p99 on this page is an optimistic number. If it tells you the budget is tight, it is tighter than that.