cost of a committee
Cost of a committee is the total overhead introduced by replacing one agent call with several coordinated agent calls. It includes all extra tokens, latency, tool use, retries, routing logic, logs and debugging effort created by the orchestration, not just the visible final answer.
Multi-agent designs are tempting because they look like added reasoning capacity: planners, reviewers, critics and specialists can divide work. The catch is that the system is still a collection of model calls. Each role needs instructions, context, intermediate messages and often tools. Without measuring that expansion, a workflow that feels more intelligent in a demo can become expensive, slow and difficult to operate in production.
The concrete way to reason about it is to draw the call graph. For each agent, count the prompt, context it receives, tool results it reads, output it produces and any retries it may trigger. Then mark which calls are parallel and which are sequential. Token cost is additive across the whole graph. Latency is governed by the longest dependent path, so parallel review is very different from a planner-refiner-critic chain.
The trade-off is not simply money. More agents can make causality harder to see: a bad final answer may come from routing, stale context, duplicated reasoning, a tool result, or a weak selection rule. Committees can also converge on the same mistake if they share assumptions or context. They help when roles are genuinely different, handoffs are narrow, and there is a reliable way to judge the outputs.
Engineers meet this cost when deciding whether to ship an orchestrated agent workflow instead of a simpler baseline. The useful artefact is a budget and trace plan before implementation: expected calls, critical path, token-bearing messages, tool invocations, retries, routing decisions and final selection logic. If those cannot be sketched clearly, the design is probably not ready to justify its operational complexity.
Common questions
- Does adding more agents usually improve answer quality?
- It depends on whether the agents add independent value. A committee can help when agents have distinct roles, separate evidence, constrained handoffs and a clear evaluation signal. It often fails when agents merely rephrase the same context, reinforce a shared false premise, or create extra intermediate text without improving the final decision.
- How should I estimate the cost before building it?
- Sketch the orchestration as a call graph. For every node, note its input context, prompt, expected output, tools, retries and downstream consumers. Sum the token-bearing inputs and outputs across all nodes. For latency, follow the longest chain of dependent calls rather than counting agents, because parallel and sequential work have very different effects.
- Why is debuggability part of the cost?
- Because multi-agent failures are distributed. The visible answer may be wrong because of an early prompt, a routing decision, a tool result, a retry, a handoff format, or the final chooser. To debug it, you need logs for prompts, outputs, tool calls, routing and selection, which adds engineering work and operational surface area.