The Harness Is the Product

“Give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime.” — proverb

One calm layered paper mold produces different stones while remaining unchanged.

I built a code reviewer. The thing I’ll actually keep isn’t the reviewer.

The reviewer is the visible product: it reads a diff, applies a rule corpus tested against real PRs and code, and ships the findings worth a person’s attention. But to get it there I had to build a second, invisible thing — the machinery that makes the reviewer good and keeps it good. Mine the rules out of review history. Tighten them against the leads the deep pass refutes. Measure recall against planted bugs. Compile the deterministic ones out to detectors. None of that machinery ships to a user. And it’s the more valuable half.

BLUF

You’re not building a reviewer. You’re building the thing that builds reviewers. The reviewer is the output; the harness — mine, tighten, measure, mechanise — is the factory.
The harness is domain-agnostic; the rules are domain-specific. Point the same machinery at a back-end codebase, re-mine the operands, and you should get a back-end reviewer out of the same tools. The verb transfers; the nouns get re-sourced. (That cross-domain transfer is the post’s bet — designed, not yet shipped; flagged again at the close.)
It’s the operator/operands split, one level up. Build the Operator keeps the apply-a-rule skill fixed and the rules as data. Here the operator is the harness — the skill of making a reviewer — and the operands are a whole domain: its rules, its eval fixtures, its detector configs, its history. Same shape, bigger unit.
It compounds across time, not just across domains. Re-run the levers over newer PRs and newly-added rules and the same reviewer keeps improving. The harness is a flywheel, not a one-off scaffold.
This reframes the moat. Rules Are the Alpha says the rules are the moat — true, per domain. The harness is the cross-domain moat: it’s what mints the next opinionated corpus cheaply. The rules are the alpha; the harness is the factory.
Honest scope: the harness is real and running (mining, tightening, evals, detectors). Repointing it to a new domain is designed, not done — I’m arguing from the shape of the tools, not a second reviewer I’ve shipped.

The visible product and the invisible one

Watch what actually ships. A user of the reviewer sees findings on a diff — that’s the product. What they never see is everything that made those findings trustworthy: the loop that mined the rules out of years of review comments, the optimiser that tightened each rule against the leads the deep pass kept refuting, the eval that plants known bugs to measure whether recall held, the detectors that took the mechanical rules out of the model’s way. That machinery runs at build time, not review time. It never touches a user’s PR.

And it’s most of the work — the split I’ve lived building it, not an economic law I can point to. The rules — the opinionated corpus everyone points at as the asset — are comparatively cheap to re-source once the machinery exists. Building the machinery the first time is the hard, slow part. Which means the durable thing I made isn’t the reviewer, and isn’t even quite the rules. It’s the factory that produces and maintains them.

One honest bound on “comparatively cheap to re-source,” because cheap is not the same as unattended. I’ve since handed most of the rule authoring to the machine — drafting, tuning, compiling detectors — and the cost that survived isn’t any of those. It’s a handful of upstream calls the machine gets wrong while producing something that looks entirely correct: naming a mechanism that already exists, re-weighting an objective, amending the rule instead of narrowing the code, changing the substrate. The factory does make the rules now. Someone still has to be standing at the right end of it, and that part hasn’t got cheaper. (Receipts in Rules Are the Alpha.)

The harness is the operator, one level up

If you’ve read Build the Operator, this is that idea moved up a level. There, the operator is the skill of applying a rule and refuting yourself; the operands are the rules, kept as editable data. The point was: train the verb, keep the noun as data.

The harness is the same move on a bigger unit. The operator is now the skill of manufacturing a reviewer — mine, tighten, measure, mechanise — and it doesn’t care what domain it runs on. The operands are a whole domain: front-end rules mined from front-end history, or back-end rules mined from back-end history. A back-end reviewer isn’t a new machine. It’s the same operator with new operands. “Train the verb, keep the noun as data” becomes “build the harness, keep the domain as data.”

Across domains, and across time

There are two axes of reuse here. Keep them apart.

Across domains is the obvious one: point the harness at a new codebase and re-source the operands — and that’s more than the rules. A new domain needs its own eval fixtures and oracles to measure recall against, and its own detector configs to mechanise the deterministic checks; all three get re-mined from the new domain’s history. What doesn’t change is the harness itself — the skill of producing and tuning that operand set. That’s the whole claim: re-mining a rules file alone wouldn’t hand you a back-end reviewer, but re-running the harness over back-end history produces the rules, the fixtures, and the detectors together. The corpus is new; the factory is not.

Across time is the one that’s easy to miss. Re-run the levers over newer PRs, and over rules you add later, and the same reviewer stays current as the codebase and the standards drift out from under it. That’s the flywheel — the harness isn’t scaffolding you tear down after launch, it’s one you keep running.

One honest bound on the flywheel, because it isn’t perpetual motion: on a frozen corpus it stalls. Once the detectors do most of the finding, the stream of refuted leads the tightening feeds on thins to nothing — a dead faucet. What keeps it fed is exactly the two inflows above: new PRs and new rules. The flywheel runs on the flow of real new work, not on itself.

One reusable harness remakes rules, evals, and detectors for new domains and refreshes them as each domain changes over time. — The reusable asset is the machinery that remakes the operands for a new domain and refreshes them as that domain changes.

It isn’t even specific to code

The clearest second instance is the one that made this post. I built a writing gauntlet while doing an earlier long-form series: mine the real receipts, run the draft past substance and voice panels and a judge from a different model family, take the real hits, and — for the pieces that need it — delete the draft and rewrite from memory so only the load-bearing parts come back. It didn’t retire when that series shipped. Every piece of writing I’ve done since has gone through it, the two books included; this post went through the whole loop, blind rewrite and all — a from-memory rewrite went up against this version in a blind, different-family judge, and this is the one that won. The harness pointed at itself. A reviewer and an essay share nothing as products. The thing that makes each one good is the same kind of asset: built once, then pointed at whatever’s next.

What this does to the moat

Rules Are the Alpha argues the rules are the moat, not the model. I still think that’s right — per domain. But it undersells the asset. A competitor who copies your rules file gets one domain’s corpus, frozen at the moment they copied it — a snapshot with an expiry date, the day your codebase moves on and theirs doesn’t. What they don’t get is the thing that produced that corpus and will produce the next one, and keep both current. Copying the output doesn’t copy the thing that produces the output. The rules are the alpha; the harness is the factory that mints alpha on demand.

Someone has now put numbers on a narrower version of that asymmetry. SkillOpt — a Microsoft-affiliated paper from May 2026 — treats an agent’s skill file as external state of a frozen model and optimises it the way you’d train weights: an optimizer proposes bounded edits to one document, a held-out split accepts an edit only on a strict score improvement, and what ships is a single file. Run that optimizer in place and it was best or tied in all fifty-two (model, benchmark, harness) cells they measured — fifty-two settings, the machinery re-run in each one. Re-running the harness works. Then they moved the file somewhere it wasn’t built for — a smaller model in the same family, or a different agent harness — with no re-optimisation. The transplanted skill kept anywhere from a tenth to 1.4 times the gain it produced at home. Copying what the harness produced pays out somewhere in that range, and you don’t learn where until you measure it in the new setting.

The floor is the part that cuts against me: “no row falls below the target’s no-skill baseline.” On every row they tested, a copied artifact was worth more than nothing. So the frozen snapshot a competitor walks off with isn’t inert — it’s unreliable. That’s a weaker claim than the one I just made, and the one I’d rather defend.

So the moat has two layers. The rules are the deep, narrow one — specific, opinionated, hard to fake for a given codebase. The harness is the wide one — it’s what lets you open a new front cheaply, the designed-not-done bet this whole post rests on. If anything, my wager is it’s the bigger asset, precisely because it’s the more transferable one. That’s a complication to “the rules are the alpha,” not a retraction: the rules are still where the per-domain edge lives; the harness is where the next edge comes from.

What you keep

Ship the reviewer. Keep the harness. The findings are the output a person sees and acts on; the machine that makes and maintains the reviewer is the asset that doesn’t leave the building. When the model becomes a commodity — the market thesis of Rules Are the Alpha — the durable thing isn’t the reviewer either. It’s the factory that can build the next one, in a new domain, next quarter, and keep the last one honest as the code moves.

You’re not building a reviewer. You’re building the thing that builds reviewers.

How sure I am: the harness is lived — mining from review history, tightening rules against refuted leads, recall evals against planted bugs, and a mechanical-detector tier are all real and running, genericised here. The operator/operands framing is my inference, settled in my own design notes, not a published result. Designed, not done: repointing the harness to a new domain (a back-end reviewer from the same machinery) is argued from the shape of the tools, not a second reviewer I’ve shipped — treat that as the load-bearing bet. The across-time flywheel is real but fueled: on a frozen corpus the drop stream dries up, so the flywheel depends on new PRs and new rules, not on itself. The SkillOpt numbers are someone else’s measurement of a different transfer than my bet — their axes are model scale, harness, and benchmark, all within one model family (GPT-5.4, its mini and nano variants, and GPT-5.5); mine is a whole domain. Their own limitations section says held-out evaluation stays necessary “before transferring them to substantially different models, harnesses, or task settings.” So it corroborates the shape of the harness-versus-artifact asymmetry. It does not test the cross-domain repointing this post rests on.

Sources

Build the Operator, Keep the Operands as Data — the operator/operands split this generalises; the harness is the operator one level up.
Rules Are the Alpha — the per-domain moat this reframes: rules are the alpha, the harness is the factory that mints them.
Keep the Factory, Ship the Result — the sibling argument (keep the generator, ship the warranted result); this post is what “keep the factory” turns out to be worth.
Yang et al., SkillOpt: Executive Strategy for Self-Evolving Agent Skills (2026, arXiv:2605.23904) — a frozen agent’s skill document optimised in text space. Run in place, SkillOpt is best or tied in all 52 (model, benchmark, harness) cells; transplanting the artifact with no re-optimisation retains 10% to 140% of its in-domain gain, and “no row falls below the target’s no-skill baseline.” Transfer tested across model scale, harness, and benchmark — within one model family.