Tracing a run
Tracing a run is recording an agent execution as a structured tree of related steps, including inputs, outputs, tool calls, model calls, timings, errors, and metadata. It lets an engineer inspect a completed or failed run later and reconstruct how the agent moved from request to result.
Agent failures are often not visible in the final answer or in a single exception. The bad behaviour may start in a prompt, a retrieved document, a tool argument, a parser, a retry, or a model response that looked plausible at the time. Flat logs can show fragments, but they rarely preserve the causal path. Tracing exists so someone who did not watch the run happen can still read the execution afterwards.
A trace models the run as a parent operation with child spans. The parent is usually the user request or job. Child spans represent concrete work such as building a prompt, calling a model, invoking a tool, parsing output, fetching context, retrying, or returning the response. Each span records its inputs, outputs, status, duration, and identifiers that connect it to the session, code path, prompt, model, and environment.
The main trade-off is that useful traces contain sensitive and bulky data. Prompts, documents, tool inputs, and model outputs may include private information, so teams need redaction, sampling, retention rules, and access control. There is also runtime and storage overhead. The honest answer to what should be captured is: enough to explain behaviour, but not so much that tracing becomes a privacy risk or an unbounded data sink.
Engineers meet run tracing when debugging agent frameworks, production incidents, evaluation failures, and regressions after changing prompts, tools, models, or retrieval code. A trace viewer typically lets you expand nested runs, filter failed or slow spans, compare metadata, and inspect the exact intermediate values. The common misunderstanding is that tracing is just prettier logging. Its value is the preserved structure of cause and effect.
Common questions
- What is the difference between tracing a run and normal logging?
- Logging usually emits individual messages from code as text or events. Run tracing records the execution as a connected hierarchy, so a model call, tool call, retry, and final response can be read in context. Logs can support tracing, but without parent-child structure and captured inputs and outputs, the cause of an agent failure is often still guesswork.
- What should be included in an agent run trace?
- Include the user request, prompts, model calls, tool calls, retrieved context, intermediate decisions, retries, errors, outputs, timings, and metadata that links the run to code, configuration, prompt version, model version, user session, and environment. Exclude or redact data that violates privacy, compliance, or retention requirements. The trace should explain behaviour, not capture everything indiscriminately.
- Does tracing change how the agent behaves?
- It should not change the agent’s logic, but instrumentation can add overhead and can accidentally affect behaviour if it changes timing, error handling, or data passed between components. Treat tracing as an observation layer: wrap calls, record boundaries, and preserve identifiers without altering inputs or outputs. Test it under realistic conditions, especially around retries and tool failures.