12.02 · Concept
What to log when the output is not stable
Log the inputs, the tool calls and the decisions, not just the final answer.
Unstable agent output needs a reconstructable trace: original and normalised input, retrieved context, memory, selected tools, tool arguments, tool results, retries, fallbacks, model choices and final output. Structured logs with correlation IDs make it possible to compare runs and separate input, retrieval, tool, prompt and model failures.
What this lesson answers
- what should I log for unstable agent outputs
- how to debug inconsistent AI tool calls
- why final answer logs are not enough
Notes
When an AI system gives unstable answers, the final response is the least useful thing to log by itself. You need the trace of what the system saw, what it chose to do, and what happened at each step: user input, normalized input, retrieved context, selected tools, tool arguments, tool results, model decisions, retries, fallbacks, and final output. The goal is to make a surprising answer reconstructable after the fact.
A good mental model is debugging a distributed request. You would not log only the HTTP response body; you would keep request IDs, upstream calls, parameters, status codes,…
Common questions
- What should be logged when an AI agent gives inconsistent answers?
- Log the request path, not only the response. Capture the user input, any normalised form, retrieved context, relevant memory, prompt inputs, tool selection, tool arguments, tool results, retries, fallbacks, model decisions and final output. The aim is to reconstruct what the system knew, chose, called, received and returned.
- Why is logging only the final AI answer not enough?
- The same bad answer can come from different causes. Input parsing may have failed, retrieval may have missed a document, memory may be stale, the wrong tool may have been chosen, arguments may be malformed, a tool may have timed out, or the prompt may have changed. Final output alone cannot separate these cases.
- How should agent logs handle sensitive data?
- Agent logs should preserve debugging value without carelessly storing secrets or personal data. Use redaction for sensitive fields, avoid raw payloads where summaries or identifiers are enough, limit retention, restrict access and sample high-volume traces when appropriate. Observability should help diagnosis without becoming an uncontrolled data store.