09.01 · Lecture
Sampling and Tails
State a probabilistic model properly: the sample space, the probability law, and what it means to assign a probability to an event.
Probabilities for failures only mean something after you define the possible outcomes, the events you will measure, and the rule assigning probabilities to those events. That structure lets request samples, probe results, error counts, and tail latency claims become precise enough to test.
What this lesson answers
- how to define probability model for failures
- what is a tail probability in reliability
- why p99 estimates need enough samples
Notes
Sampling and Tails — Sampling and tail probabilities exist to make statements about rare failures mathematically testable; without a sample space , probability law , and events , claims like “the system usually works” or “timeouts are rare” have no precise meaning.
Key Concepts: - A probabilistic model is the triple where is the set of possible outcomes, is the set of measurable events, and satisfies .
Common questions
- What makes a probability statement about production failures well defined?
- You need to say what one outcome is, which sets of outcomes count as events, and what probability rule applies. For example, a request succeeding or failing is not enough on its own. You also need the population, window, endpoint, and assumptions behind repeated observations.
- Why are tail probabilities useful for deployment reliability?
- They express rare but operationally important cases, such as unusually slow requests or bursts of errors. Averages can hide these behaviours. Tail probabilities let you ask whether extreme outcomes are expected under the model, rather than relying on vague statements like timeouts are rare.
- What goes wrong when failures are treated as independent by default?
- The model can understate risk when failures share a cause. Retries, overloaded sidecars, shared dependencies, or regional incidents can make observations correlated. If the independence assumption is false, simple count models give misleading probabilities for consecutive failures or large error bursts.
Short definition: what is Sampling and Tails?