10.02 · Concept
A test set for something non-deterministic
Assemble cases that catch regressions without pretending the output is stable.
Non-deterministic agent tests should assert behaviours, constraints and failure modes, not exact strings. A useful regression set captures real incidents, edge cases and core journeys, then judges whether each run remains acceptable through deterministic checks, rubrics, review or model-based scoring.
What this lesson answers
- how to test non deterministic LLM outputs
- what should an agent eval case contain
- how to catch regressions in LLM agents
Notes
For an agent or LLM feature, a test set is not a list of expected strings. The same prompt can produce different valid answers, and tiny model, prompt, retrieval, or tool changes can alter wording without breaking the product. The goal is to assemble cases that represent behaviors you care about, then check whether each run still satisfies those behaviors well enough to ship.
A useful mental model is a production incident suite, not a compiler unit test suite. Each case should describe the input situation, the task, any constraints, and the properties that make an answer acceptable or…
Common questions
- Can you write regression tests for an LLM if outputs vary?
- Yes. The test should not expect identical wording. It should check stable product behaviours: whether the answer uses the right facts, follows constraints, calls required tools, avoids forbidden content and handles the user’s task correctly. Variation in phrasing is acceptable when those properties still hold.
- What belongs in an agent evaluation test case?
- A good case describes the user situation, the task, relevant context, required constraints and unacceptable outcomes. It also states how the result will be judged. Some cases can use deterministic assertions, while others need a rubric, reference answer, human review or an LLM judge.
- Where should an LLM regression test set come from?
- Start with production incidents, known edge cases and important user journeys. These cases represent behaviours the product must preserve or improve. Include examples where the agent previously failed, places where safety or policy matters, and paths that are commercially or operationally critical.