Replace the Trivial, Augment the Difficult
A rule in the code-review engine I built caught an idiom I've been writing for years and can no longer see. That's the small half.
A rule in the code-review engine I built caught an idiom I've been writing for years and can no longer see. That's the small half.
A change lands in review with everything green. The rules pass, the types check, the tests are close to the ones I'd have asked for. I read the diff anyway, and the thing I'm looking for isn't in it.
The request was: let me arrange these panes how I like, give the arrangement a name, and hand it back to me tomorrow on a different laptop.
One evening an agent working in this repo killed an idea of mine for failing a rule. The kill was well argued and it cited the rule by line number, the way the standards file demands. The next morning I opened the file at that line.
It looks at a deprecation deadline declared in a schema, calls new Date(), compares, and if the deadline has passed it fails. Eighteen rules in that plugin; that's one of them. It's published as a library and other repos consume it.
My reviewer caught a deleted flush call three times out of three, and said nothing about eight of thirteen removals in any run. A check that works a quarter of the time doesn't present as a gap. It presents as a feature.
A newer engineer left a note on my pull request: this array should be readonly. He was right.
Sixteen minutes and sixteen dollars to surface four low-severity nits, most of it spent scanning for rules rather than judging code. A corpus can fit the context window and still overflow the model's attention.
I spent a stretch trying to make an AI code reviewer cheaper without letting it miss more. Cost and recall, two dials pulling against each other.
The reflex, when a model meets something long, is to make it shorter. Summarise the PR, TL;DR the paper, have the agent review the diff and hand back the findings. It all points one way: read less.
Ask a code-review tool how good it is and you'll get a precision answer: of the things it flagged, how many were real. That's easy — every flag is a claim you can check.
The goal was never a cheaper reviewer. It was finding everything that is really there while posting nothing that isn't — and that objective, not the budget, is what forced the pipeline apart.
A while back, I stopped reading the AI's end-of-task summaries and started watching the diffs. They came through faster than I could have typed them. I wasn't reading them line by line — I was watching the shapes go by.
A description is a compression, and it drops the omissions it doesn't know it made — the case not handled, the check not added. It can't surface a gap it never noticed, which is where the bugs are.
Linus Torvalds vibe-coded the visualizer and hand-wrote the audio math. One repository, the whole argument — writing code was never the scarce skill.