Skip to content
Klay

Read the diff, not the summary of the diff

Read the actual diff before you merge it: the summary is a claim, the diff is the fact.

EngineersVibe codersgithubgitlab

You own code you did not write, from the moment you merge it. A generated summary describes what the change was intended to do, which is the one thing you already knew — it cannot tell you about the line that was also changed, the test that was quietly relaxed, or the error branch that was deleted because it looked unused. Those are only visible in the diff.

What goes wrong: A change that does what the summary said AND three other things nobody mentioned, one of which is a regression.

What agents change: A model writes an extremely convincing summary of its own work, including the parts it got wrong — the summary and the code come from the same place.

You are violating it when

  • You merge on the strength of a description of the change.
  • You cannot say what a merged change did to files outside its stated scope.
  • Test files in a diff go unread.

The usual objection: That reviewing the intent is reviewing the change. Intent is the part you can verify in conversation; the diff is the part you cannot verify any other way.

A pull request description, commit message, or generated recap is useful orientation, not review evidence. The reviewable artifact is the set of changed lines: added branches, deleted checks, edited tests, modified configuration, and quiet changes in nearby files. Merging means accepting all of those changes, including the ones nobody called out.

This works because regressions often hide in mismatches between the stated purpose and the actual edit. A summary can say the change adds validation while the diff also weakens an assertion, removes an error path, or changes a default. Reading the diff forces the review to cover the code path that will run, not just the story about why it was changed.

The misconception is that reviewing intent is reviewing the change. Intent can be discussed with the author, inferred from the ticket, or summarized by a tool; it is not the executable reality. The diff is the only place where accidental edits, overbroad refactors, and test dilution reliably appear.

With an agent, the summary deserves even less special trust because it is produced by the same system that produced the patch. It may confidently describe the intended fix while overlooking the same mistaken assumption embedded in the code. Agent-written changes therefore need ordinary diff review, often with extra attention to scope and to whether each changed line is necessary.

Install it

npx klay practices add read-the-diff-not-the-summary
  • .github/pull_request_template.mdcreate
    ## What this changes

<!-- One or two sentences. What behaviour is different after this merges? -->

## Why

<!-- The decision, not the mechanism. What were the alternatives, and why this one? -->

## How it was verified

<!-- Say what you actually ran. "Tests pass" is not verification; name the case. -->

- [ ] I have read the full diff, not only the summary of it
- [ ] Every file in this diff is one I meant to change

The previews are the first lines of each file; the command writes them in full. Existing files are never overwritten.

How you know it stuck

npx klay practices audit reports this check for this practice:

  • pr-template-present

Where this comes from

  1. What to look for in a code reviewGoogle Engineering Practices · Official docsEstablishes the reviewer’s responsibility for every changed line, independent of the author’s explanation.
  2. Navigating a CL in reviewGoogle Engineering Practices · Official docsAdds the concrete review behavior: deliberately traverse all files so omitted edits are seen.
  3. Helping others review your changes - GitHub DocsGitHub Docs · Official docsDefines self-review as inspecting the diff for accidental changes before asking others to trust it.
  4. Agent pull requests are everywhere. Here's how to review them.GitHub Engineering · Engineering blogExtends the rule to agent-authored pull requests and end-to-end logic tracing.
  5. Complacency with AI-generated code | Technology Radar | ThoughtworksThoughtworks Technology Radar · Engineering blogExplains why AI-generated change sets make summary-trusting more tempting and more dangerous.

Questions

Reading every diff does not scale.
Correct, which is why the previous practice caps how big they are. The two rules only work as a pair.