AI Wrote More Code. Who Reviews It?
My multi-agent AI code review system finds, verifies, and ranks issues before I open the pull request.
Get the guide to build your first AI agent directly in your inbox on newsletter signup:
Last week, one of my AI reviewers found that an implementation was missing fields required by JSON-RPC.
I had reviewed the code. What I had not done was read enough of the standard to know which fields were absent.
I have enough experience as a software engineer to read a function and know its design can be improved. But I can’t start using a technology and understand its “gotchas” immediately. That’s the problem AI is solving for me now.
AI has made producing code much cheaper. It has not created more time for me to inspect that code. When I am involved in several projects, I cannot deeply read every pull request, compare every implementation with the previous version, check every library contract, and still do my own work.
Yet I got positive feedback about being able to contribute in multiple workstreams instead of having the typical approach of putting my head down in my work and ignoring everything else.
GitHub defines AI code reviews as an automated process that analyzes code and provides feedback. That definition is correct, but too broad to be useful on its own. A generic “review this PR” prompt usually produces a generic review.
My system is more specific:
Several reviewer agents inspect the same change from different starting points.
Each finding must include evidence.
One aggregator verifies and combines the results.
Repeated findings become deterministic build-time checks.
The review runs before I’m even aware I have to review the PR
The agents can review several projects in parallel. I still work on one thing at a time. Let me tell you how I do this.
AI moved the bottleneck from writing code to reviewing it
A few days ago, The Pragmatic Engineer described a concern among engineering leaders: coding agents are producing more code, review load keeps growing, and some developers are reviewing changes less thoroughly.
I have experienced the same pressure. AI makes writing code faster, so we have to work on more projects in parallel.
This creates two bad situations.
The first is approving faster. The tests are green, the AI reviewer did not leave a blocking comment, and the diff looks reasonable. Ship it.
The second is making a senior engineer the permanent review queue. That may keep the quality bar high for a while, but it does not scale. The engineer spends the day switching between PRs while multiple authors wait.
AI code review tools can help, but only if it reduces the work required to understand a change. The term AI slop comes from having a lot of noise for little signal. A wall of low-confidence comments creates more work for humans.
Cloudflare described exactly this failure when it put a diff into a broad prompt. The output included vague suggestions, hallucinated errors, and comments about code that already existed. They ended up with a system that used up to seven specialist reviewers and one coordinator.
That architecture matches the lesson from my own experiments. Let’s see now what my architecture is
A review agent should make the code easier to judge. It should not make the reviewer process another stream of guesses.
One reviewer agent == one responsibility
You may have heard that getting 1% better every day leads to getting 35 times better over a year. That’s the power of the compound effect. Also, the same principle under the butterfly effect: A small change in one place can make a big difference later.
Well, large language model (LLM) outputs are probabilistic and sensitive to their starting context, so they are sensitive to the butterfly effect.
The first files an agent opens influence the model it builds of the change. A reviewer that starts from a controller may focus on request validation. A reviewer that starts from a database migration may focus on compatibility and rollback. Both can be the smartest LLM, both can have the same codebase; we have and still miss what the other one sees.
Humans do this too. Once I think I understand the cause of a bug, my attention starts looking for evidence that supports that explanation. Another engineer can enter the same review with a different mental model and find the gap I ignored. That’s why I always worked in teams where, besides the author, we had 2 human reviewers.
One general reviewer has an even harder job. Think about it like a senior engineer that is reviewing the PRs of 20 people, but it’s not working in any of those projects. We can ask it to inspect security, product behavior, tests, architecture, performance, documentation, and team conventions inside one long prompt. Attention gets split across concerns; it can pattern-match with different projects, but it can’t go deep.
For me, this is the reviewer roster that I want to have:
Security reviewer: Checks all security concerns
Behavior reviewer: Compares the change with the ticket, API contract, and existing behavior.
Test reviewer: Looks for untested branches, weak assertions, missing failure cases, and tests that merely repeat the implementation.
Architecture reviewer: Checks ownership boundaries, coupling, dependencies, cross-package impact, and following the team convention.
Standards reviewer: Compares the code with repository conventions, library documentation, language rules, and external standards.
These reviewers should be independent. I do not want the architecture reviewer to inherit the security reviewer’s conclusions before it has inspected the change. Separate starts create separate opportunities to notice a bug.
The good part of bugs is that it’s not a matter of all reviewers aligning that we should fix something. Once one reviewer finds something and proves it, it shares it, and an aggregator adds it to the list of requested changes.
A review comment is useful only when it brings evidence
Every agent output has to answer these questions
Location:
Which file and line contain the problem?
Finding:
What behavior is wrong or risky? Is it only about design / team conventions?
Why it matters:
What can fail, regress, leak, or become harder to maintain?
Evidence:
Which parent-commit behavior, related code, test, documentation, library contract, language rule, or standard proves the claim?
Suggested next step:
What is the smallest useful correction or decision?
Confidence:
How certain is the reviewer, and what remains unknown?
This is how I get one report, categorized and prioritized, and I can evaluate as a last human step whether we should write a comment in the PR about it or not.
LogRocket’s tool comparison used separate prompts for security, context alignment, and modularity. That is already better than asking for “a complete review and make no mistakes.” The part I add is independent reviewers plus an aggregator, letting each agent find conclusions independently.
The evidence field may be the most important part of the AI code review system.
Start with the parent commit. A reviewer should distinguish a regression introduced by this pull request from an old problem that happens to be nearby. If the same behavior existed before, it may deserve a separate ticket instead of blocking the current change.
Then inspect related merged code. A new API client may be inconsistent with the other clients in the repository. A new retry loop may ignore the backoff helper every other package uses. The comment becomes stronger when it points to the established implementation.
For libraries, languages, and protocols, use the authoritative documentation.
That is how the JSON-RPC issue I opened the article with was found. The valuable output was not “this response is missing id”. The reviewer identified the location, named the missing fields, and connected the finding to the protocol's exact section that described this. I could verify the comment by myself in just a quick read.
This also changes how I review across multiple projects. I don’t need to remember every standard and every package pattern before I open the PR. The agents collect the relevant sources. I inspect the evidence and decide whether the finding applies. This is especially important when having repositories in multiple programming languages, some legacy and with bad design, others more robust and with strong standards and conventions.
One caveat is that evidence can prove that an implementation violates a documented contract. It still cannot decide whether the team should change the contract, accept an inconsistency, or move a responsibility to somewhere else. That is engineering judgment. That is still my job.
Repeated feedback should become deterministic
Reviewer agents will eventually repeat themselves.
Maybe they keep flagging the same package naming convention. Maybe generated code calls a forbidden dependency from the wrong layer. Maybe every new endpoint forgets the same required field.
The easy response is to add another paragraph to AGENTS.md, a steering file, or the reviewer prompt. That’s a good first step, but it may not fully mitigate the problem. But they’re weak enforcement because the model can still miss, misread, or deprioritize the instruction.
I prefer this progression:
A reviewer finds a problem.
I verify that the rule is real and broadly applicable.
Fix the current instance.
We encode the property in a deterministic check.
The context file explains the convention as a best effort to do it right since the first time.
I did this recently for Java package conventions. AI kept creating packages that did not follow the team’s agreed pattern. Even with markdown files referenced in skills and prompts, it often created packages outside the convention. The durable fix was an ArchUnit rule (a Java library to define unit tests that check things like cycles in between your functions or the Java packages and class naming). Now the build fails when new code crosses that boundary. That’s a stronger feedback mechanism to AI than instructions in a Markdown file.
The same approach works with:
Formatting and naming: Linters and formatters.
Architecture boundaries: ArchUnit, dependency rules, or import restrictions.
API shapes: Schema validation and contract tests.
Security invariants: Static analysis, policy checks, and targeted tests.
Required behavior: Unit, integration, and end-to-end tests.
A green build does not prove the code is correct. It proves the properties encoded by those checks. That is still a major improvement because those properties no longer depend on a model remembering a paragraph.
This is a cool feedback loop when you are adopting AI-assisted code reviews. Reviewer agents find classes of mistakes. Humans decide which findings should become a policy. Deterministic tools enforce the policy on every future change.
The review system gets better for your team even if the model stays the same. Even after 1 month, I still didn’t switch my Opus 4.8 for Fable, just because it was so hard to find spare capacity in my company that it was falling back all the time. And because Opus 4.8 was more than enough to build a system around it.
If you want to build this feedback loop around coding agents, read the guide to harness engineering. It covers context, state, guardrails, and deterministic checks in more depth.
Conclusion: Run the review before you arrive
This kind of AI-assisted review takes time. Five reviewers plus an aggregator are slower and more expensive than one generic prompt.
So I do not sit there watching them run.
The workflow fires when a pull request is opened or updated The agents fetch the change, inspect their assigned concerns, verify their findings, and prepare one report.
This is the operating model:
PR published
Trigger my orchestrator that fires specialist reviewers
These specialist reviewers take their time
Aggregator verifies, deduplicates, and generates one report
First human contact: I open the PR and the report. I focus on this PR for as long as I need to have an opinion and leave my comments. My agents are working in the background and preparing me with other reports so I can properly switch to the next context without leaving part of my attention behind.
My agents work concurrently. My attention stays sequential.
Start small, improve it over time. That is how the review system compounds. Every PR can improve the guardrails for the next one.
If you found value in this post:
❤️ Click the heart to help others find it.
✉️ Subscribe to get the next one in your inbox.
💬 Leave a comment with your biggest takeaway
♻️ Share it with your peers
Today’s article will allow you to move from phase 2 to phase 3, gaining influence as you’re able to review more code with higher quality, getting you promoted faster
I’m building this system below for paid subscribers. Thanks for your continued support!
🗞️ Other articles people like
If you want to improve the human side of the same workflow, read this next. It shows how references, scope, and escalation make review comments easier to act on.
Apply these strategies when reviewing code (Part 2)
·Get the free AI Agent Building Blocks ebook when you subscribe.
