How is this different from simply running several tool calls in parallel?

Parallel calls only describe execution timing. Orchestrator and workers also require a manifest, stable worker IDs, per-worker status, retry accounting, and a deterministic join. The important property is not just that work happens concurrently, but that every expected item is either matched to a result or reported as failed.

Should the final answer include worker failures?

Usually yes, at least in structured form or in a concise user-facing summary. Hiding failures is the main failure mode this pattern prevents. The exact wording depends on the product: an internal operations tool may list failed IDs, while a user-facing assistant may say which items could not be processed and why.

What should a worker return to the orchestrator?

A worker should return a compact, typed result rather than its whole transcript: worker ID, status, relevant extracted fields, evidence or citations if needed, error details if failed, and cost or trace metadata where useful. This keeps the merge predictable and avoids paying to re-read long intermediate conversations.

Where should retries be handled?

Retries are usually tracked per worker, not only at the whole orchestration level. That lets one flaky item be retried without redoing unrelated work, and lets the join report the item as failed after its budget is exhausted. The right budget depends on idempotency, cost, latency limits, and the failure type.

Orchestrator and workers

Orchestrator and workers is a control pattern where a parent agent decomposes a run into identifiable child tasks, runs them independently, then joins their outputs with explicit success and failure accounting. It prevents parallel or batched agent work from producing a final answer that hides missing, errored, or unmapped sub-results.

The pattern is needed because multi-step agent traces can make lost work look like completed work. If an agent processes many documents, tickets, or tool batches in a single long run, a timeout or malformed result may be buried in the trace. Without an explicit manifest of expected work and a join that checks it, the final response can summarise only the successful parts while implying that everything was processed.

Concretely, the orchestrator creates a fan-out manifest: each input gets a stable worker identifier and a task type. Workers run as separate branches, graph nodes, or agent invocations, and each reports status, output, attempts, tool calls, and metadata. The join step matches results back to the original identifiers, computes what is missing or failed, and passes a structured set of successes and failures into the final merge.

The trade-off is extra machinery. You need identifiers, schemas, retry policy, trace propagation, and a merge function that is more disciplined than concatenating text. Parallelism can reduce wall-clock latency when workers run concurrently, but it does not make the model work free: the token and tool cost is still roughly the sum of the worker runs plus the merge. Large worker transcripts can also make the join step expensive unless outputs are compact.

Engineers meet this pattern in LangGraph branch-and-join flows, agent SDK handoffs, queue-backed job runners, and observability traces where each worker should appear as a child span. A common misunderstanding is that an orchestrator is just a planner. In this pattern it is also the accountant: it must know what was launched, what returned, what failed, what was retried, and what the user can safely be told.

Go deeper

Agents & MCPOrchestrator and workersFan work out and merge it back without losing track of what failed.Open the lesson

Common questions

How is this different from simply running several tool calls in parallel?: Parallel calls only describe execution timing. Orchestrator and workers also require a manifest, stable worker IDs, per-worker status, retry accounting, and a deterministic join. The important property is not just that work happens concurrently, but that every expected item is either matched to a result or reported as failed.
Should the final answer include worker failures?: Usually yes, at least in structured form or in a concise user-facing summary. Hiding failures is the main failure mode this pattern prevents. The exact wording depends on the product: an internal operations tool may list failed IDs, while a user-facing assistant may say which items could not be processed and why.
What should a worker return to the orchestrator?: A worker should return a compact, typed result rather than its whole transcript: worker ID, status, relevant extracted fields, evidence or citations if needed, error details if failed, and cost or trace metadata where useful. This keeps the merge predictable and avoids paying to re-read long intermediate conversations.
Where should retries be handled?: Retries are usually tracked per worker, not only at the whole orchestration level. That lets one flaky item be retried without redoing unrelated work, and lets the join report the item as failed after its budget is exhausted. The right budget depends on idempotency, cost, latency limits, and the failure type.

Every term