Rules Are the Alpha

“Tell me what you like, and I’ll tell you what you are.” — John Ruskin, Modern Painters

Two reviewers, the same commit — a fat change, a few thousand lines. One was a deep reviewer I own, running a portfolio of judgment rules tested across a whole corpus of real PRs and code. The other was a shallow general reviewer: read the diff, flag what looks wrong, no encoded standard behind it.

The deep one came back with about twenty findings, overwhelmingly code-quality: an unlocalised string, a type widened where it didn’t need to be, a switch that wasn’t exhaustive, a lookup that had gone dead by accident and nobody noticed. Maybe one honest bug in the pile. The shallow one came back with about seven, all surface — and one of them was a layout regression the deep reviewer walked straight past.

That gap is the whole post. It’s not a clean ablation — the two reviewers differed in more than their rule sets, so I can’t pin it on rules alone — but the shape is hard to unsee: the depth on the left tracks an encoded standard a broad read can’t reach, and the miss on the right tracks a lens I never wrote a rule for. That’s the pattern the rest of the post builds on.

BLUF: Bottom line up front

• The model is a commodity; the rules are the alpha. Anyone can rent the same frontier capability. What they can’t rent is your company’s encoded opinion of good code.
• The alpha isn’t that you have rules — it’s how they’re made and structured. “Rules are the moat” is a saturated claim now; every reviewer ships a rule feature. Three things are actually defensible: where the rules come from, whose opinion they hold, and how they’re kept out of the model’s way.
• Mined, not written. The corpus comes from the company’s own review history — its accumulated judgment — not a blank editor and good intentions.
• Opinionated, which a product structurally can’t be. Anything sold to everyone has to be opinion-free to sell. That confines it to the rules nobody argues with. The contested calls — the ones that make your codebase yours — are exactly what a universal product can’t ship. Alpha can’t be a product.
• Mechanised, so a big corpus doesn’t drown the model. Too many prose rules smear the model’s attention (lost in the middle) into mush; compiling the deterministic ones out of context fixes it. The how is its own piece — A Big Rule Set Is an Attention Problem.
• Automating the rule-writing didn’t drain the alpha — it moved the human upstream. The machine drafts and tunes most of my rules now. My input didn’t shrink; it relocated to four moments — naming a mechanism that already exists, re-weighting an objective, amending the rule versus narrowing the code, changing the substrate. Review catches none of them, because all four would have shipped a perfectly plausible artifact.
• Honest ceiling: the list gives recall only over what’s on it. The un-ruled surface bug still gets through — see the layout regression above. And a rule judges the code that’s there, so a diff that deletes a guard or an await leaves nothing for any rule to point at — a class no longer list reaches.

Rules are a genre now, so that’s not the claim

Start by conceding the obvious. “Your rules are the moat, not the model” is not an insight anymore — it’s a genre. The self-serve reviewers market encoded institutional knowledge; several pitch learning from your repo; “your agent is only as good as its rules” is on landing page after landing page (CodeRabbit, Qodo, Greptile, and the rest of a crowded category). The premise underneath — that the base model is the commodity, and value accrues to what you build on top — is now mainstream enough that a coalition of ~25 companies pressed a version of it on U.S. policymakers in a July 2026 open-weights letter — an interested position, and one the closed-frontier labs didn’t sign. Microsoft’s own CEO made the same case in a July 2026 essay: enterprises pay twice for AI — in tokens, then in the know-how they leak back through prompts and corrections — so you own the learning loop (data, traces, evals, adapted weights) and keep the model swappable underneath (Nadella, The Reverse Information Paradox). When a company the size of Microsoft is arguing the model is the commodity, that half isn’t the insight anymore. If the claim were just rules exist and they matter, or just the model is a commodity, a reader would rightly close the tab.

So the claim is narrower, and it survives the concession: the alpha is in how they’re made and structured — three properties, none of which falls out of handing someone a rule editor: provenance, stance, mechanisation.

Mined, not written

A rule you type into a config box is a guess about what you care about. A rule mined from your company’s review history is a record of what you actually enforced. Those are not the same asset, and the second one is far harder to fake.

The corpus here was built the way the good miners build — mostly harvested from the org’s own pull-request comments, the accumulated judgment of years of people saying “not like that, like this” — plus a hand-built minority for the gaps mining always leaves. This matters for a reason my own book already labours: the code cannot hold every rule. The invariants it can’t express become rules the reader has to remember, and head knowledge is just debt. Review comments are where that debt gets called in — where a human says the unwritten rule out loud to another human. Mining the comment history is harvesting exactly the rules the code could never carry.

That’s why “we curate, they mine” is the wrong frame — both mine. The difference is whose comments. Mining one org’s opinionated history yields a coherent point of view. Mining a recurrence-gated team average across everybody yields the mean, and the mean is mid: the argument being that teams trained on the consensus drift toward the same standard and lose what made their codebase singular. A coherent opinion is the thing that resists mean-reversion. That is what makes it alpha rather than beta.

It has a second payoff: the rules are the fixed point across model churn — keep them as an explicit corpus and you swap the model underneath without re-deriving what you care about. (Why per-version drift makes that matter is You Can’t Grade Your Own Misses.)

Opinionated, which is why it can’t be a product

Here’s the structural part, and it resolves what used to look like a bragging claim.

You can’t honestly say “more rules than the competition” — nobody can measure that. But you can say something sharper and structural: a product sold to everyone has to be un-opinionated to sell to everyone. That confines it, by construction, to the rules nobody argues with — a big library of the universally agreeable. And a big library of universally-agreeable rules is still beta. It’s the mean. The opinionated rules — this pattern is banned here even though it’s fine elsewhere, we eat this cost to buy that property, this “clever” line is exactly the kind we reject — those are the alpha, and a universal product structurally cannot ship them. It can only offload them to each customer to write.

So the deepest form of the claim — and I’ll flag it as my argument, not a proven law — is that alpha can’t be a product: it’s precisely what a product can’t be. Anything off the shelf must be opinion-free, and opinion is the point. The obvious rebuttal is that a vendor could mine each customer’s PRs and hand back a tenant-specific opinion — and a vendor is now arguing it in public. A sponsored piece in The New Stack (Jain and Poll, August 2026) proposes exactly that and names it: an AI slop register, built by pulling a team’s last thousand review comments, sorting them into deterministic / execution-testable / genuine judgment, and extracting the first two buckets back into rules that become institutional memory. That is this post’s mechanism, sold as a feature — so read the rebuttal as live, not hypothetical. It still only relocates the work: building the opinionated corpus is tedious, and — my wager — most teams aren’t opinionated enough to hold a defensible view of good code in the first place. The tooling commoditises; the disposition — taste, patience, the conviction to do the hard yards — does not.

One honest flag, because the claim reads arrogant without it: an opinionated corpus encodes an opinion, not the truth. Mine your own comments and you inherit your own blind spots. That’s what the hand-built minority is for, and it’s why that minority carries more weight than its size suggests.

Mechanised, so a big corpus doesn’t drown the model

The third property is what stops a large corpus turning back into a liability. Pile a hundred prose rules into the model’s context and its attention smears across all of them — lost in the middle — and you get mush that saw every rule and internalised none. So the deterministic rules get compiled out of context into code, and the model’s scarce attention lands only on the rules that need judgment. That whole move — the attention budget, the rule structure, the exceptions loop — is its own piece: A Big Rule Set Is an Attention Problem. For this post the point is narrower: mechanisation is the third property that makes the corpus defensible — an owned standard kept out of the model’s way.

The machine writes the rules now, and the alpha moved upstream

There’s an obvious objection to all three properties: if an agent can author rules, then provenance and stance are cheap, and the moat drains. I’ve been running that experiment on myself without meaning to. Most of the rule work is automated now — drafting rules out of real findings, tuning their wording, compiling the deterministic ones into detectors, running the sweeps that decide which survive. If you want to know what a moat is made of, take a piece out and watch whether the value follows it down.

It didn’t follow. But I can’t tell you the rules stayed as good, because I can’t measure that honestly — baseline jitter on the eval I’d use is 45%, wider than most of the deltas worth detecting. The receipt is a different one, and it’s better precisely because it needs no quality measurement: I keep a log of every moment my input changed the machine’s direction. It is one-sided by construction — it records the interventions that landed, not the ones that were noise — and I’ll come back to that. Eight entries so far. They sort into four kinds, and every one of them sits upstream of the work rather than in review of it.

Naming a mechanism that already exists. The model was about to widen a shared vocabulary file so that two names for one defect would stop colliding in its measurements. I named a function in its own codebase — four words, no argument attached. Reading that function showed the product already collapsed those duplicates; the collision existed only inside the measurement harness. It had been about to churn a file other teams sync against, to fix a bug in its own instrument.

Re-weighting an objective. I’d pinned a priority order — fidelity first, then cost, then speed — and shipped it as advice in Metered the Wrong Thing. A total order reads the way a total order reads: the lower term never trades. So cost was unbudgeted, every cheap lever stayed opt-in forever, and that was correct under the rule I gave it. The fix wasn’t softening the priority. It was admitting that a strict order asserts all recall is worth the same, and it isn’t.

Amend the rule, or narrow the code? A detector implemented its rule faithfully and over-fired on a controlled-component fallback pattern that the widely-used component libraries all use. The model’s instinct was to narrow the detector until the noise stopped. The call that came back: the rule is what’s wrong. Narrow the detector instead and the real rule now lives in code, where nobody reading the rule set can find it.

Changing the substrate. The model was iterating on syntactic queries — better patterns over the same text. I asked what it would look like as a graph of data flow instead. Four of the five queries built on the new substrate died. The one that lived catches a shape the standard lint rule is structurally blind to: a value that changes on every mouse move flows into a callback, so the callback’s identity changes, so a listener tears down and re-registers on every frame — and every dependency array in the chain is correct, which is exactly why the lint rule stays silent.

What makes those four a category rather than four anecdotes is the thing they share. Every one of them would have shipped something that looked fine.

The moment	What the machine would have shipped
Naming the mechanism	A clean commit widening a vocabulary file — reviewable, mergeable, aimed at the wrong target
Re-weighting the objective	A coherent optimisation, faithfully serving the priority I wrote down
Rule versus code	A narrowed detector, green tests, no more noise
Changing the substrate	Working queries, incrementally better than last week’s

Review catches wrong output. None of these produced wrong output. That is the whole reason the job moves upstream instead of into the review queue.

I’ve argued the general shape before — the human doesn’t leave, the job moves up — but that piece argues what you should do, and this is the first time I’ve had a receipt for where the human actually went. The receipt turns out to be wider than the argument. That post lands on owning the standard, and only the second of these four is standard-setting. The other three aren’t about the target at all. They’re about which mechanism, which artifact, which substrate — and get those wrong and the machine does excellent work in the wrong place, at whatever rate you’re paying for.

So the wager two sections up survives, sharpened. The tooling did commoditise: the machine now does the authoring I used to do by hand. My input didn’t shrink with it. It relocated, to four moments where the output looked right and wasn’t.

The ceiling, said honestly

A rule list is a checklist, and a checklist beats memory for recall — you go down it and verify each item instead of hoping the reviewer notices. That’s Gawande’s argument for human checklists; extending it to an encoded rule set a model runs is my transfer, not his result — though a small check backs it. On synthetic diffs with planted rule-violations, a 7B reviewer handed the rule checklist caught about 75% of them, against roughly 12% for a bare “review this”; an irrelevant checklist changed nothing (~4%), so it’s the specific rules doing the work, not a generic “here’s a list” nudge. (Small: n=6, one model, and the checklist also raised false positives — harness and numbers in the repo.) It also makes misses attributable: “rule R didn’t fire” is a fact you can act on, where a silent human miss is just silence.

But enumeration only covers what’s enumerated. The rule set is this reviewer’s ceiling for rule-driven recall — systematic within the list, and beyond it you’re relying on something else: the model’s own read (the logic pass) or plain luck, the way the shallow reviewer stumbled onto the layout bug. Systematic recall stops at the edge of the list; that isn’t the same as total recall. Which is exactly the layout regression from the opening: the deep reviewer had no lens for the rendered result, so no rule fired, so it sailed past a bug the shallow reviewer caught by accident. That’s not a weak reviewer. It’s a missing rule.

And some misses aren’t a missing rule at all. A whole class — logic bugs — comes from the model reading the code and noticing what looks fragile, not from any rule firing: my reviewer’s logic pass runs on the model’s weights and a few “this looks risky” cues, with no rulebook behind it. That’s a different asset — the irreducible read — and the subject of the Reading series (Give Away the Writing; A Description Can’t Replace the Code). Rules are one kind of alpha; the reading is the other.

There’s a third kind of ceiling, and it’s the one I can’t argue away by writing another rule. A rule judges the code that’s there. But a diff that deletes something — a call, a guard, an await, a catch block — usually leaves head code that reads perfectly fine. Delete await store.flush() and the surviving lines are unobjectionable. There’s nothing to point a rule at, because the defect isn’t in the code; it’s in the change. No rule fires, and no head-code rule could — it would have to be a rule about an absence, and an absence has no line to anchor to.

That isn’t a gap in my list. It’s a class the rule form structurally can’t express, and the only pass that can see it is the one that reads base against head. I got a rough read on how well the model catches these unaided by inverting my planted-bug fixtures — flipping base and head so each diff becomes a removal of the behaviour the bug had added. The reviewer flagged the removal in about a quarter of runs, and only where the deletion was loud: a dropped flush, a swallowed error. On eight of the thirteen it said nothing, every time. Treat that as a smell test, not a recall figure — I haven’t adjudicated which of those removals actually warranted a comment, and some plainly don’t (reverting an off-by-one fix restores correct code). What survives the caveat is the shape: it notices the dramatic deletions and is quiet on the rest. So the honest ceiling has three layers, not two: the rule you never wrote, the reading no rule replaces, and the defect with nothing for a rule to point at. More rules fixes only the first.

So how does the first layer grow? Not by someone remembering to write a rule. Every finding my reviewer emits carries a free-text field — outOfListFactors — for the things that pushed the call but weren’t on the rule’s condition list. That reads like a leak in an enumerated system, and it is the opposite: it is the only place the system can tell you what you failed to enumerate. Read the field across runs and the recurrences are the candidate conditions. Three came out of mine. An incident number cited in the comment turned up three times as the thing that made a borderline finding worth keeping. “Explains a bug you couldn’t derive from the code” turned up three times. A five-line block clearing the bar turned up twice, where the rule’s condition implied a three-line window — the threshold was wrong, and the field is what said so. Each recurrence gets promoted into an explicit condition, and the escape valve narrows by exactly that much. The valve isn’t a hole in the enumeration. It’s the mechanism the enumeration grows by.

This is the positive mirror of Evals Aren’t the Moat: evals reach only what you thought to specify, and the moat is the judgment they can’t see — the rule corpus is that judgment made into a durable asset instead of a hope. Getting a rule out of a reviewer is its own problem (Teaching the Reviewer); this post is about what you have once it’s out: an encoded, opinionated, mechanised standard a rented model and a blank rule editor can’t reproduce between them.

The model is the commodity. The rules are the alpha — because of where they came from, whose opinion they hold, and how they’re kept out of the model’s way.

---

How sure I am: the asset is lived — the rule portfolio exists, tested against a whole corpus of real PRs and code, mostly mined from the org’s review history with a hand-built minority; the two-reviewer receipt is a real run, genericised. Checked, small-n: the checklist-beats-bare transfer — a laptop pilot (Qwen2.5-7B, n=6) put rule-checklist recall at ~75% vs ~12% bare and ~4% for an irrelevant checklist, so it is the specific rules, not priming; directional, one model, harness in the repo. Argued, with a smell test behind it: that a deletion leaves nothing for a head-code rule to anchor to is an argument from the rule form, not a measurement. The inverted-fixture run (thirteen fixtures, three runs, removals flagged in ~10 of 39) is a real run but an unadjudicated corpus — I have not decided which removals deserved a comment, so read it as the shape of the gap, not its size. The out-of-list harvest is a real mechanism running on real findings, but the three promoted conditions came off recurrence counts of three, three, and two — enough to show how the enumeration grows, not enough to claim a rate at which it does. Not done and marked as such: the model trained to apply the corpus — designed and argued, not built or measured. “Alpha can’t be a product” is my framing of a publicly-arguable idea, not sourced to any company or memo — though the rebuttal to it now has a named public instance, a vendor-sponsored piece proposing the mine-and-codify loop as a product feature (Sources) — and the corpus encodes an opinion, with its own blind spots, which is why the hand-built minority matters. The upstream section, bounded three ways: the intervention log is one-sided by construction — it records where my input changed direction, and I’ve asked for the cases where it was wrong or a wash and haven’t got them yet, so read the four moments as a taxonomy of what worked, not a hit rate. One of the four is still only half-landed: the re-weighted objective references a per-arm severity the measuring instrument can’t currently produce, so that correction has outrun the machinery it was meant to steer. And there is no measured claim that rule quality held when the machine took over the authoring — baseline jitter on that eval is 45%, wider than most deltas worth detecting, which is exactly why the section rests on where my input landed instead of on a quality hold. (The mechanisation/attention claims and their prior art live in the sibling piece.)

Sources

• Atul Gawande, The Checklist Manifesto (2009) — checklists beat expert memory for recall; the enumeration argument for a rule list.
• The AI-code-review category (CodeRabbit, Qodo, Greptile, and others) — the “encoded institutional knowledge / learns-your-repo” marketing that makes “rules are the moat” a genre. Named as landscape evidence; I’m characterising public positioning, not auditing each product.
• How to Think About Code, the implicit-rules chapter — rules the code can’t express become rules the reader must remember; head knowledge is debt. Backing for mining review comments (the author’s own book).
• Open Weights and American AI Leadership — coalition letter, ~25 signatories (NVIDIA, Microsoft, Meta, and others; hosted by NVIDIA), July 24 2026 — argues open weights broaden access and that value accrues to specialisation, data, and ownership on top of the base model. Cited only as external corroboration that “the model is the commodity” is now a mainstream industry framing — a public position, not proof, and an interested one (the closed-frontier labs OpenAI and Anthropic did not sign).
• Ankit Jain and David Poll, Code review is a taste problem (The New Stack, August 13 2026) — a sponsored, contributed piece (the site’s own taxonomy tags it sponsor-aviator, sponsored-post-contributed) arguing review’s remaining jobs are collaboration, alignment and verification, and proposing an “AI slop register”: pull the team’s last thousand review comments, split them deterministic / execution-testable / genuine judgment, and codify the first two buckets into rules that become institutional memory. Cited as the named public instance of the mine-each-customer’s-PRs rebuttal in Opinionated, which is why it can’t be a product — an interested position, and note that the split it offers (“around 45/30/25”) is hedged in the original (“you might end with,” “may be codifiable”) and carries no corpus, method, or n.
• Satya Nadella, The Reverse Information Paradox (July 2026 essay; reported in The New Stack, TNW) — enterprises “pay twice” for AI (tokens, then leaked know-how); own the learning loop and keep the model a swappable commodity. More mainstream-industry corroboration that the model is the commodity — and, like the letter, an interested position: Microsoft sells the orchestration layer that framing routes the value to.

---

Related: A Big Rule Set Is an Attention Problem (how you wield a large corpus without drowning the model — the mechanisation half of this post); Teaching the Reviewer (how you get the rule out of yourself in the first place); Evals Aren’t the Moat (the negative mirror — judgment is the moat, evals reach only what you specified); You Can’t Grade Your Own Misses (per-version drift — why the rules outlive the model). Part of the reviewer-rules cluster with Build the Operator, Keep the Operands as Data (train the apply-skill into the weights, keep the rules as data) and Train on What the Gate Throws Away (turn the reviewer’s rejects into training data).