10.01 · Concept
Why grading the answer is not enough
Explain how an agent reaches a right answer the wrong way, and why that matters.
A correct final response can hide an unsafe or unreliable agent run. Agents make intermediate choices: tool calls, retrievals, retries, observations and handoffs. Evaluating only the output misses whether the answer was grounded, whether constraints were respected, and whether the same behaviour would hold up in production.
What this lesson answers
- why is final answer grading insufficient for agents
- how can agents get right answers incorrectly
- what should agent trajectory evaluation check
Notes
For ordinary software outputs, checking the final answer is often enough: did the function return the expected value, did the API produce the right JSON, did the page render correctly. Agents are different because they take multiple steps: they choose tools, read intermediate observations, revise plans, and may interact with external systems before producing an answer. Two agents can return the same final text while one followed a safe, reliable path and the other guessed, used the wrong source, leaked private data, or performed unnecessary destructive actions.
The useful mental model is to…
Common questions
- Why is checking the final answer not enough for an agent?
- Because an agent can produce the expected text after using bad evidence, the wrong tool, private data, or an invalid shortcut. The output tells you what it said, not how it got there. For production systems, the path matters because it affects reliability, security, compliance, cost and debuggability.
- What is an agent trajectory?
- An agent trajectory is the record of the steps taken during a run: decisions, prompts, tool calls, retrieved material, observations, retries and handoffs. It is the operational trace behind the final answer. Reviewing it shows whether the agent’s behaviour was justified, safe and repeatable, not merely whether the last response looked correct.
- What can go wrong if an agent gets lucky?
- A lucky success can mask a behaviour that fails on the next similar request. The agent may have guessed, ignored constraints, relied on leaked information, or used evidence that does not support the answer. If you only grade the result, that weakness remains invisible until it causes a harder-to-debug production failure.