Skip to content
Klay

AI review is a second reader, never the reviewer

Let an AI reviewer comment first, and never let it be the only approval on a change.

EngineersVibe codersgithubgitlab

An automated reviewer is genuinely good at the tireless things — the null you forgot, the copy-paste that kept the old variable, the missing await. It is bad at the thing review exists for: deciding whether this change should be made at all. Worse, it approves plausible-and-wrong code with the same tone it approves correct code, which is precisely the failure a reviewer is there to catch.

What goes wrong: Code written by a model, approved by a model, merged by a human who assumed someone had thought about it.

What agents change: When the author is a model and the reviewer is a model, nobody in the loop has ever been accountable for the outcome.

You are violating it when

  • A pull request's only approval came from a bot.
  • Reviewers reply to the bot's comments and not to the diff.
  • Nobody on the team can explain why a merged change was needed.

The usual objection: That an AI reviewer is a weaker human reviewer. It is a different instrument: much better at breadth, structurally incapable of judgement about intent.

An AI reviewer is useful as an early, tireless pass over a change. It can flag mechanical mistakes, suspicious patterns, missing checks, and inconsistencies before a person spends attention on the pull request. That makes the later human review cleaner, not optional.

The mechanism is separation of jobs. Automated review is strongest at breadth and repetition; human review is where intent, product fit, architecture, risk, ownership, and tradeoffs get judged. A model can sound equally confident about a correct change and a plausible wrong one, so its comments need validation by someone accountable for the outcome.

The misconception is that an AI reviewer is just a weaker human reviewer. It is not. It is a different instrument: often faster across many local details, but not a responsible participant in deciding whether the change belongs in the system.

When an agent also wrote the code, the risk compounds. A model-created patch followed by model approval can create the appearance of review without any accountable judgement. The human who sends or merges the change remains the first real owner of whether it should exist.

Install it

npx klay practices add ai-review-is-a-second-reader
  • .github/workflows/klay-ai-review.ymlcreate
    # Klay practice: ai-review-is-a-second-reader
# https://klaylearn.com/practices/ai-review-is-a-second-reader
#
# Posts an automated review as a COMMENT on the pull request. It deliberately
# does not approve, does not request changes, and cannot satisfy a required
# review.
#
# That restraint is the whole practice. An automated reviewer that can approve
# closes the loop with no human in it: a model writes the change, a model
# approves the change, and nobody was ever accountable for whether it should
# have been made. This workflow is a second pair of eyes, first — not the pair
# of eyes.
#
# Requires: gh CLI (present on GitHub runners) and a model API key.

The previews are the first lines of each file; the command writes them in full. Existing files are never overwritten.

How you know it stuck

npx klay practices audit reports this check for this practice:

  • ai-review-in-ci

Where this comes from

  1. About GitHub Copilot code review - GitHub DocsGitHub Docs · Official docsFirst-party product guidance frames AI feedback as incomplete and requiring human validation.
  2. Application card: GitHub Copilot Agents - GitHub DocsGitHub Docs · Official docsResponsible-use limits from the vendor document misses on complex or large changes.
  3. What to look for in a code reviewGoogle Engineering Practices · Official docsDefines human review around design judgement, not only defect detection.
  4. Agent pull requests are everywhere. Here's how to review them.GitHub Engineering · Engineering blogPlaces responsibility for agent-authored changes on the human opening the pull request.
  5. Complacency with AI-generated code | Technology Radar | ThoughtworksThoughtworks Technology Radar · Engineering blogNames the complacency failure that turns automated confidence into reduced scrutiny.

Questions

We are a team of one. Who is the second reader?
You are, tomorrow morning. Let the bot comment immediately and read the diff yourself before merge — the point is two passes with different strengths, not two payroll entries.