What is an LLM judge?

An LLM judge is a model prompted to assess another model’s output. It may score an answer, choose between alternatives, apply a rubric, flag problems or explain its judgement. It is most useful where exact assertions are too brittle, but its output should be treated as an evaluator signal rather than objective truth.

Why can LLM judges be unreliable?

They can reward fluency, length, confidence or familiar style instead of correctness. They may miss subtle factual errors, be swayed by prompt wording, prefer outputs from similar model families or accept reasoning that only appears coherent. Without evidence and calibration, the judge’s mistakes can look deceptively authoritative.

How should I make LLM grading safer?

Use a clear rubric, include task context, add reference material or evidence for factual checks, and compare the judge against human-labelled examples. For pairwise comparisons, randomise answer order. Inspect disagreements, keep held-out cases, and monitor behaviour over time so drift and systematic bias are caught early.

Evaluating agents

10.03 · Concept

LLM as judge, and its failure modes

Use a model as a grader while knowing what it is systematically bad at.

An LLM judge is a noisy evaluator for outputs that resist exact tests, such as usefulness, tone, reasoning quality and task completion. It can be useful with rubrics, references and evidence, but it is not ground truth. Treat its scores as one signal, then calibrate, audit and monitor its error profile.

What this lesson answers

how to use LLM as a judge
when do LLM judges fail evaluation
how to calibrate an LLM grader

Notes

Using an LLM as a judge means asking a model to score, compare, or critique another model’s output. This is useful when the thing being evaluated is hard to check with exact tests: helpfulness, tone, reasoning quality, faithfulness to a document, or whether an agent completed a task sensibly. The mental model is not “the judge knows the truth”; it is “the judge is a noisy evaluator whose behavior must itself be evaluated.”

A practical setup usually gives the judge a rubric, the task input, the candidate answer, and sometimes a reference answer or retrieved evidence.

Common questions

What is an LLM judge?: An LLM judge is a model prompted to assess another model’s output. It may score an answer, choose between alternatives, apply a rubric, flag problems or explain its judgement. It is most useful where exact assertions are too brittle, but its output should be treated as an evaluator signal rather than objective truth.
Why can LLM judges be unreliable?: They can reward fluency, length, confidence or familiar style instead of correctness. They may miss subtle factual errors, be swayed by prompt wording, prefer outputs from similar model families or accept reasoning that only appears coherent. Without evidence and calibration, the judge’s mistakes can look deceptively authoritative.
How should I make LLM grading safer?: Use a clear rubric, include task context, add reference material or evidence for factual checks, and compare the judge against human-labelled examples. For pairwise comparisons, randomise answer order. Inspect disagreements, keep held-out cases, and monitor behaviour over time so drift and systematic bias are caught early.

Short definition: what is LLM as judge, and its failure modes?

10.02 · Evaluating agentsA test set for something non-deterministic 10.04 · Evaluating agentsGrading the trajectory