11.05 · Concept
Budgets and circuit breakers
Cap what a single run can spend before it spends it.
Run-scoped budgets stop an agent before repeated calls, retries or loops turn into uncontrolled cost and latency. Treat each run as carrying a remaining allowance, check it before expensive work, and use a circuit breaker to fail safely, downgrade behaviour or return a fallback when the allowance is exhausted.
What this lesson answers
- how to limit LLM agent spending per run
- how to stop runaway model calls in agents
- why timeouts are not enough for LLM budgets
Notes
A budget is a constraint you enforce before work happens, not a report you read afterward. For LLM systems, the spend of a run comes from repeated model calls, token volume, tool calls, retries, and loops. A circuit breaker is the piece of control logic that stops the run once a limit is near or reached, returning a controlled failure or fallback instead of letting the system continue spending money and time.
The mental model is the same as production rate limits and timeouts: every run carries a remaining allowance. Before each expensive action, the system estimates or accounts for its cost,…
Common questions
- What is a circuit breaker for an LLM agent?
- It is control logic that stops or changes execution when a run is about to exceed an allowed cost, call count, token use, retry count or time limit. Instead of letting the agent continue looping or retrying, it returns a controlled error, partial result, confirmation request or cheaper path.
- Why not just review traces or billing after the run?
- Post-run observation tells you what already happened. It cannot prevent a faulty agent loop from making repeated expensive calls during the run. A useful budget guard is enforced before each costly action, so the system can deny, downgrade or stop work while there is still something to protect.
- Is a timeout enough to control agent cost?
- No. A timeout limits elapsed time, but a fast loop can still issue many model or tool calls before the clock expires. Cost control needs separate caps for the expensive resources involved, such as model calls, token volume, retries and estimated spend, checked as the run progresses.
Short definition: what is Budgets and circuit breakers?