What is the control loop in autoscaling?

An autoscaler repeatedly reads a metric, compares it with a configured target, and adjusts capacity. The metric is the controlled signal, the target is the setpoint, and the actuator is whatever changes serving capacity, such as pod replicas, VM group size, or container instance count.

Why can autoscaling make a service unstable?

Instability usually comes from delay and overcorrection. Metrics are not instant, new capacity takes time to become useful, and a strong reaction to a short spike can add too much capacity. When the metric later drops, the scaler may remove capacity again, creating oscillation.

Is CPU always the right metric for autoscaling?

No. CPU works when CPU demand tracks user-visible pressure, but it is poor for many I/O-bound services. If requests are slow because workers are waiting on a database, CPU may look healthy while latency is bad. Queue depth, concurrency, lag, or latency can be better signals.

Scale, Cost, and Limits

11.04 · Lecture

Autoscaling as a Control Loop

Recognise the general structure of a feedback system - measure the output, compare it to the target, correct - which is the structure an autoscaler implements.

Autoscaling is a feedback controller: observe a service metric, compare it with the desired state, then change capacity through replicas, instances, or concurrency. The hard parts are choosing the right signal, accounting for delay, avoiding overcorrection, and knowing when added workers will simply move the bottleneck elsewhere.

What this lesson answers

how does autoscaling work as a control loop
why does Kubernetes HPA oscillate under bursty traffic
which metric should an autoscaler target

Notes

Autoscaling as a Control Loop — Autoscaling exists to keep a service near a target such as $60%$ CPU or $p 95 < 300 ms$ by repeatedly measuring load, comparing it to the target, and changing capacity; without this feedback loop, traffic spikes cause overload while quiet periods waste money.

Key Concepts: - A feedback loop has 3 steps: measure output, compare to a target, then correct using an actuator; for autoscaling this is typically metric $\to$ desired replicas $\to$ add/remove pods, VMs, or instances.

References

Kubernetes - Horizontal Pod Autoscaling

Common questions

What is the control loop in autoscaling?: An autoscaler repeatedly reads a metric, compares it with a configured target, and adjusts capacity. The metric is the controlled signal, the target is the setpoint, and the actuator is whatever changes serving capacity, such as pod replicas, VM group size, or container instance count.
Why can autoscaling make a service unstable?: Instability usually comes from delay and overcorrection. Metrics are not instant, new capacity takes time to become useful, and a strong reaction to a short spike can add too much capacity. When the metric later drops, the scaler may remove capacity again, creating oscillation.
Is CPU always the right metric for autoscaling?: No. CPU works when CPU demand tracks user-visible pressure, but it is poor for many I/O-bound services. If requests are slow because workers are waiting on a database, CPU may look healthy while latency is bad. Queue depth, concurrency, lag, or latency can be better signals.

Short definition: what is Autoscaling as a Control Loop?

11.03 · Scale, Cost, and LimitsScaling Axes 11.05 · Scale, Cost, and LimitsEconomics, and When to Leave