12.01 · Concept
Tracing a run
Capture every step of an agent run so a failure can be read afterwards.
Agent tracing records a run as a structured tree of model calls, tool invocations, prompts, retrieved context, retries, errors, timings and metadata. It turns a failed execution into something inspectable after the fact, so the cause can be found without relying on scattered logs or memory.
What this lesson answers
- how to trace an agent run
- what should agent tracing record
- why logs are not enough for agents
Notes
Tracing an agent run means recording the execution as a structured story, not just saving logs. An agent run may include user input, prompts, model calls, tool calls, retrieved documents, intermediate decisions, retries, errors, outputs, timings, and metadata. The goal is that someone who was not present during the failure can later reconstruct what happened and where the behavior diverged from expectations.
A useful mental model is a distributed trace for reasoning. The top-level run is the user request, and each child span is one step the agent took: planning, calling the model, invoking a…
Common questions
- What is agent run tracing?
- Agent run tracing is the practice of recording an agent execution as connected steps rather than isolated log lines. The top-level run represents the user request, while child spans represent planning, model calls, tool calls, parsing, retries and final output. Each step carries enough context to reconstruct what happened later.
- What should I capture in an agent trace?
- Capture the user input, prompts, model and tool calls, retrieved documents, intermediate decisions, retries, errors, outputs, timings and relevant metadata. Useful metadata includes identifiers for the session, code path, model version, prompt version and environment. Avoid recording sensitive data or unnecessary payloads that increase risk and cost.
- Why are normal application logs insufficient for debugging agents?
- Normal logs are usually flat and inconsistent, so they make it hard to follow cause and effect across prompts, model responses, tools and retries. Agent failures often occur inside intermediate reasoning, malformed tool arguments or poor retrieval results. A trace preserves the execution structure, making the failure path readable afterwards.
Short definition: what is Tracing a run?