Top-Down vs Bottom-Up Evals for Self-Improving Systems

In Self-Improving Systems: Proof, Not Promises, we argued that a system claiming to improve itself has to show its work: replay logs, agreement rates, pre-registered bars, the whole evidence trail. This piece continues that thesis by digging into the evaluation layer itself, because "show your work" only means something if you're measuring the right thing in the first place.

Most teams building self-improving systems pick one of two eval philosophies and stop there. Either they define correctness up front, from intent, and grade every change against that definition. Or they let production data define correctness after the fact, and grade every change against what actually happened. Both are legitimate. Both are also, alone, a way to fool yourself. The question isn't top-down vs bottom-up evals as a choice. It's how you wire them together so each one catches what the other misses.

Two Different Questions, Not Two Versions of the Same Question

Top-down evals start with intent. Before you write a line of code, you decide what "correct" looks like: a hand-built eval set, unit tests, invariants, conservation properties, a pre-registered success bar. Then you measure the system against that fixed target. The question top-down answers is: does the system do what we meant?

Bottom-up evals start with the world. You capture what actually happened in production, human corrections, edit logs, override patterns, and you treat that history as labeled data. You replay candidate changes against it before they ship. The question bottom-up answers is: does what we meant survive contact with reality?

These are not the same question, and a system that only answers one of them is only half-verified. A model can pass every test you wrote and still fail the world you didn't imagine. A model can match historical human behavior perfectly and still be faithfully reproducing yesterday's mistakes. You need both, and you need them to talk to each other, not sit in separate dashboards.

Top-Down Evals: Crisp, Cheap, and Quietly Overfittable

The appeal of top-down evals is obvious. You write down what correct means before you're tempted to rationalize what the system actually did. A pre-registered bar is a commitment device: you can't move the goalposts after you see the score, because you already told everyone where the goalposts were.

Top-down evals are also cheap to run. A 16-case eval set, a battery of unit tests, an invariant check like "nothing lost or invented across the pipeline" runs in seconds and gives you a clean, defensible number. For catching regressions against a known spec, nothing beats it.

But a fixed eval set has a failure mode that looks exactly like success while it's happening: overfitting to the eval, not to the underlying behavior you actually care about. We watched this happen directly. Iterative prompt tuning against a fixed 16-case eval set produced a monotonic decline in real-world quality: 96.1, then 80.6, then 72.2, then 69.4. Each step, in isolation, looked like a reasonable response to the eval's feedback. Tune the prompt, rerun the 16 cases, see the number move, ship it. The problem is that a 16-case eval set has almost no room in it. Once you've optimized against every quirk of those 16 cases, further "improvement" against the eval is just memorizing its specific shape, at the cost of everything the eval didn't happen to cover. The eval score and the real-world score decoupled, and the eval score kept looking fine right up until it didn't.

This is the structural weakness of top-down evals: they measure your imagination of the world. If your 16 cases don't include the edge case that shows up in production on a Tuesday, your eval will tell you the system is fine while it's wrong in production. A fixed target, iterated against long enough, becomes a target you can hit without actually solving the problem.

None of this means top-down evals are the wrong tool. It means they're the wrong only tool. Invariants, conservation checks, and pre-registered bars are indispensable precisely because they're rigid: rigidity is what makes them trustworthy. The fix isn't to loosen them. It's to pair them with something that measures the part of the world they can't see.

Bottom-Up Evals: Correctness Harvested From What Actually Happened

Bottom-up evaluation flips the source of truth. Instead of deciding correctness in advance, you harvest it from the world: every time a human corrected the system's output in production, that correction is a labeled example of what "right" looked like in a situation you didn't have to imagine, because it already happened.

In one implementation, on an order-operations pipeline, we backfilled 135,052 labeled exemplars directly from historical human corrections, reconciled to within 1 row of an independently derived oracle. That's not a curated eval set someone wrote by hand; it's the accumulated residue of real operators fixing real problems, at a scale no hand-built test suite gets close to. From there, every candidate model change gets replayed against 16,000-plus model-vs-human pairs before it ships, not after. And critically, more than 3,500 of those pairs are zero-edit confirmations: cases where the human looked at the system's output and changed nothing. That's a signal too. Silence, in a system where humans routinely correct mistakes, is evidence of correctness, and treating only edits as data throws half the picture away.

The strength here is obvious: bottom-up evals measure the world as it actually arrives, not as you predicted it would arrive when you wrote your test cases. They scale with production instead of with an engineer's afternoon. They catch the failure modes you didn't think to write a unit test for, because they're built from cases where reality already surprised somebody.

But bottom-up evaluation has its own structural weakness, and it's the mirror image of top-down's: it's retrospective. Agreement with the past tells you the system reproduces the past. It says nothing about whether the past was right. If human operators had a systematic blind spot, or a workaround that was itself a mistake, a system that matches their historical corrections perfectly will faithfully encode that mistake as ground truth. High agreement with history can mean you've built something excellent or something that quietly automates an old bug. The number alone can't tell you which.

There's a second problem: labels drift. What counts as "the human's correction" isn't always a stable fact. When we re-labeled a batch of exemplars using first-edit instead of latest-edit as the ground truth, 153 corrections shifted, 0.52% of all pairs. That's a small percentage, but it's a real one, and it's a reminder that bottom-up ground truth is not bedrock. It's itself a modeling choice: which edit counts, how much time elapses before a correction is "final," whether a reversal three days later invalidates the original label. Bottom-up evals are honest about the world, but the world is messy, and the eval inherits that mess.

Where This Breaks If You Only Use One

Run the thought experiment forward on each philosophy alone.

Top-down only: you write your invariants, your unit tests, your 16-case eval set, and you optimize against them indefinitely. You will get a system that is provably correct against your intent and quietly wrong against reality, because your intent was written by someone who couldn't see every case in advance. The 96.1 to 69.4 decline is what this looks like in practice: a system getting worse while its own scoreboard says it's improving.

Bottom-up only: you harvest 135,052 exemplars from history and optimize agreement with them. You will get a system that reproduces human behavior at scale, including whatever the humans got wrong, including whatever labeling ambiguity crept into the 0.52% of pairs that shift depending on which edit you trust. You have no external check on whether "matches history" and "correct" are the same thing, because you never defined correct independently of history.

Both failure modes are dangerous specifically because they don't look like failure while they're happening. Top-down failure looks like a clean, improving eval score. Bottom-up failure looks like high agreement with a large, real dataset. Neither philosophy contains, on its own, a mechanism for catching its own blind spot. That mechanism only exists when you put both in the same loop and let one check the other.

How They Work Together: Top-Down Invariants Bound the Bottom-Up Replay

Here's the operating pattern, drawn from the order-operations pipeline referenced above, generalized to its shape rather than its specifics.

Conservation proofs gate the replay corpus. Before any bottom-up comparison is trusted, a top-down invariant runs first: nothing lost, nothing invented, across the pipeline. That check ran against 1,500 replayed inputs from the bottom-up corpus and confirmed conservation held. This is top-down doing what it's best at, a rigid, cheap, unambiguous check, applied directly to the bottom-up data before that data gets to do any evaluative work. If the corpus itself were silently dropping or duplicating records, every agreement number computed from it downstream would be meaningless. The invariant isn't a competitor to the replay; it's the precondition for trusting it.

A pre-registered bar decides what "worked" means, before you see the result. An adversarial-checker experiment was evaluated against a bar set in advance, and it failed: AUC 0.47, blind, which is worse than a coin flip. The reason that failure is a useful data point instead of a rationalized footnote is entirely due to pre-registration. If the success bar had been set after looking at the result, 0.47 would have quietly become "directionally interesting" or "worth more investigation." Because the bar was fixed first, the failure was visible, unambiguous, and actionable: the experiment was shelved. This is what top-down evaluation is for. It's not there to be generous. It's there to make failure impossible to talk your way out of.

Bottom-up replay catches what the top-down bar can't see, then top-down gates the ship decision. A learned-rule change went through replay against the human-edit corpus before it shipped: 96% agreement with what humans actually did, and the replay surfaced 2 rule gaps that got caught pre-review, not in production. That's bottom-up evaluation doing its job, finding real cases a hand-written test suite wouldn't have thought to include. But the decision to ship still ran through an explicit, top-down ship/no-ship threshold. The bottom-up evidence didn't ship itself. It fed a pre-registered gate that decided, independent of how good 96% might feel in the moment, whether the number cleared the bar.

Replay validates a design change before it goes live, and the shipped result confirms the prediction. A replay against historical outcomes predicted an 86% flip rate for a proposed design change, meaning the new logic would produce a different outcome than the old logic in 86% of historical cases where the two disagreed. That's a strong claim to make before shipping anything. The change went live, and the shipped result matched the predicted flip rate. This is bottom-up evidence doing something top-down evals structurally can't do on their own: validating that a change grounded in stated intent actually behaves the way its designers predicted, against real historical volume, before it's live in front of customers.

Notice the pattern across all four: bottom-up supplies the coverage and the scale; top-down supplies the discipline that keeps that scale from being mistaken for truth. Conservation proofs check the corpus. Pre-registered bars check the experiments. Ship/no-ship thresholds check the replay results. None of these interventions are exotic. They're some of the plainest tools in software verification. What makes them work here is that they're applied at the seam between the two philosophies, not just inside one of them.

The Practical Version of This for Your Own System

If you're building a self-improving system and you want to know whether your eval strategy has a blind spot, ask these questions in order.

First: do you have a pre-registered success bar for every experiment that matters, written down before you see results? If the bar gets set or adjusted after you've seen the number, you don't have a top-down eval. You have a story.

Second: is your bottom-up ground truth actually large and structured enough to mean something, or is it a handful of anecdotes you're calling "production feedback"? 16,000 model-vs-human pairs is a dataset. Twelve support tickets is not, no matter how compelling they are individually.

Third: are you capturing zero-edit cases as signal, or only logging when a human intervenes? If you only track corrections, you're throwing away every confirmation, and your agreement rate is measuring a biased subset of production, not production itself.

Fourth: do you have an invariant, however basic, that checks the integrity of your bottom-up corpus itself before you trust any comparison run against it? Conservation checks, row-count reconciliation, schema validation, anything that verifies the data you're replaying against wasn't corrupted or incomplete on the way in.

Fifth: when your top-down and bottom-up evals disagree, what happens? If the answer is "we go with whichever one looks better," you don't have a verification system. You have two number generators and a bias toward optimism.

Closing Frame

Top-down tells you the system does what you meant. Bottom-up tells you whether what you meant survives contact with reality. Neither is optional, and neither is sufficient. Optimize against a fixed eval set alone and you'll watch your real-world quality erode while the scoreboard climbs, the same way a 16-case eval walked a system down from 96.1 to 69.4 in four moves that each looked fine. Optimize against historical agreement alone and you'll build something that reproduces the past faithfully, mistakes included, with no independent check on whether the past deserved reproducing.

Put them together, with top-down invariants gating what the bottom-up replay is allowed to trust, and pre-registered bars deciding what counts as shipped, and you get something neither philosophy delivers alone: a system whose claims about its own improvement are actually falsifiable, and that failed falsification attempts, like an AUC 0.47 experiment, get surfaced instead of buried.

That's the evidence loop. It's not more complicated than running two eval philosophies. It's just less comfortable, because it means your system has to survive being wrong in front of you, on purpose, before it ships.

If you're building operational software that needs to prove its improvements rather than assert them, this is the kind of evaluation architecture we build into the systems we deploy. Book a discovery call and we'll walk through what top-down and bottom-up evals would look like for your specific pipeline.