11.03 · Short-concept
Scaling Axes
Compare the common load balancing algorithms - round robin, least connections, hashing and the rest - and say what each one optimises for.
Load balancing algorithms choose which backend gets each request, and each choice optimises for a different failure mode: even counts, unequal capacity, long-lived work, latency, cache affinity, or low coordination cost. The right algorithm depends on whether your bottleneck is CPU, connections, tail latency, locality, or replica churn.
What this lesson answers
- round robin versus least connections load balancing
- when to use consistent hashing in load balancing
- what load balancing algorithm optimises for latency
Notes
Scaling Axes — Scaling axes exist to choose how requests are distributed across replicas; without the right load-balancing algorithm, systems break through hotspots, uneven latency, lost cache locality, or overloaded long-lived connections.
Key Concepts: - Round robin sends request to backend , optimizing for simple even request counts when requests have similar cost, e.g. 1,000 equal HTTP requests across 10 pods gives about 100 requests per pod. - Weighted round robin assigns backend a weight and targets share , e.g.
References
Common questions
- When is round robin the right load balancing algorithm?
- Round robin is a good default when backends are roughly identical and requests cost about the same. It spreads request counts evenly without needing live load measurements. It performs badly when request duration, CPU cost, or connection lifetime varies significantly, because equal request counts can still create overloaded replicas.
- Why use least connections instead of round robin?
- Least connections is better when work stays open for different lengths of time, such as WebSocket traffic or other long-lived sessions. It sends new work to the backend with less active connection pressure. It is less reliable when a single connection can multiplex many concurrent requests, as with HTTP/2 or gRPC.
- What problem does consistent hashing solve?
- Consistent hashing keeps most keys mapped to the same backend when replicas are added or removed. That matters for cache locality, sticky routing, and stateful affinity. Plain modulo hashing can reshuffle most keys during scaling, which causes cold caches, uneven latency, and avoidable load spikes.
Short definition: what is Scaling Axes?