Your Team's Routing Habits Are the Spec

The floor noticed before we did. That's the part that still bothers me.

We'd shipped an automatic release step for one large reseller's orders, part of a larger order routing automation project meant to cut the manual approval queue down to the orders that actually needed a human. The release logic worked. It matched what we'd tested. And within days it had sent a run of that reseller's approved orders past a production step that almost every other order goes through, because the automation didn't know what every experienced approver on that account already knew without being told: this customer supplies ready-to-print art. Skip the step. Every time. It's not a special case to them, it's just how that account works.

Nobody had written that down. It lived in the approvers' heads, in the way they clicked through the queue, in the account name they recognized on sight and routed on instinct. The fleet-wide default didn't know it existed, so it ran every order the same way, and the reseller's orders went to a station they'd never needed and didn't want.

Here's the position, stated plainly before we get into the mechanics: if you're building order routing automation for an operation with any account history at all, the fleet-wide default is not a safe starting point. It's a guess that happens to be right for whichever customers look like your average customer, and wrong for every account that doesn't. The fix isn't more testing before launch. It's mining your own approval history for the account-level patterns your staff already learned, and making the account-level rule the default before the fleet-wide one ever touches that account's orders.

The unwritten rule is not an edge case

Call it what it is. When one account's orders almost always get approved a certain way, that's not noise sitting on top of a general process. That's the actual process for that account. The general process is the noise, in the sense that it describes what happens to a customer nobody has a specific rule for.

Experienced staff carry dozens of these. Account A always supplies print-ready art, skip the proof step. Account B's purchase orders never match the requested ship date, don't flag the mismatch, it's normal for them. Account C is an affiliate of Account D, route them the same way even though the emails come from different domains. None of this is written in a manual. It's written in the approval log, one decision at a time, by people who've been doing this long enough to stop thinking about it consciously.

That's exactly why it's invisible to a build process that starts from requirements documents and stakeholder interviews. Ask an operations manager to describe the routing logic and you'll get the general rule, the one that applies to most orders. Ask them to enumerate every per-account exception and they'll tell you the top three or four they can remember off the top of their head. The rest live in the muscle memory of whoever's been clicking approve on that account's orders for years, and muscle memory doesn't show up in a spec document. It shows up in the log.

Why the default hides the exception until it breaks

A fleet-wide default doesn't fail loudly. It fails quietly, for exactly the accounts it wasn't built for, and it can take days before anyone notices, because the failure doesn't look like a system error. It looks like an order sent to the wrong place, which on a busy floor can look like a lot of things before it looks like a routing bug.

That's what happened with the reseller. The automation didn't crash. It didn't throw an error. It processed the order exactly the way it processes every order, which was the problem: it treated a distinctive account like an average one. The general rule is correct for most orders by construction, since it's built from the aggregate. That's precisely why it's wrong for the accounts sitting furthest from the aggregate, and those are usually your biggest, oldest, most idiosyncratic accounts. The ones with the most history. The ones your staff have the most unwritten rules about.

There's a pattern underneath this that shows up across every automation project we've touched, not just routing: the deployment lesson isn't "the model made a mistake." It's "the model didn't know a rule that a human held tacitly and never got asked to state." We've written before about why human-in-the-loop review doesn't automatically catch this kind of gap either (https://aiterated.com/chronicles/human-in-the-loop-is-not-a-safeguard), because a human reviewer approving one order at a time doesn't necessarily notice that the automation is running a different rule for an entire account until enough orders pile up wrong. The floor caught this one because someone downstream saw a physical order arrive at the wrong station. That's a fortunate failure mode. Not every misroute is that visible.

Mining the approval log for the real rules

The fix isn't smarter general logic. It's asking a narrower, more mechanical question of your own history: for each account, does the approval pattern differ from the fleet-wide pattern, and by how much?

Concretely, that means going back through the approval log and segmenting decisions by account instead of looking at them in aggregate. Where does one account's approve/reject/route pattern diverge sharply from what the same order would get under the general rule? You're not looking for every account, most of them will track the fleet-wide default closely enough that no override is needed. You're looking for the accounts where the divergence is consistent, not a one-off. A single unusual approval could be a mistake or a special request. A consistent pattern, order after order, month after month, is a rule someone is following even if they've never said it out loud.

For the reseller in question, that pattern was almost total and easy to find once we asked the right question of the log: staff routed their approved orders past the production step, essentially every time, because the account always supplied ready-to-print art. Not most of the time. Almost always. That's the signature of an account-level rule, not a coincidence.

This is the same logic we use across order routing automation and every other operational workflow we build: applied software logic means modeling the business logic chain that's actually running the operation, not the one written in a policy document. We've laid out that distinction in more general terms here (https://aiterated.com/chronicles/business-logic-chains-explained), and it holds just as well for routing as it does for order entry or ship-date estimation. The chain that matters is the one your staff execute, not the one your org chart implies.

Once you've found the pattern, the fix is not complicated. You don't need a smarter model or a bigger training set. You need an account-level override that fires before the fleet-wide default gets a chance to apply, scoped tightly to the accounts where the log actually supports it. For the reseller, that meant a default at the account level: route approved orders past the production step, always, unless something in the order itself says otherwise. No fleet-wide reasoning, no general heuristic trying to infer intent from the order contents. Just the rule the humans were already running, made explicit and made the default for that one account.

The replay is what tells you whether you got it right

Finding the pattern and writing the override is the easy half. The harder question, and the one that actually matters for shipping this kind of change safely, is: did the fix only change behavior for the account it was meant to fix, or did it quietly change behavior somewhere else too?

You don't answer that by reasoning about the code. You answer it by running the new rule against historical orders across every account and checking, order by order, whether the outcome changed for anyone it shouldn't have. That's a replay, and it's the only way we've found to be confident that an account-level fix is actually account-level and not a broader rule wearing a narrow name.

For the reseller fix, the replay showed exactly what you'd want it to show: the new default changed the outcome for that one account's orders, the ones it was built for, and changed nothing for any other account. No other customer's approved orders started skipping the step. No adjacent account with a similar name or similar-looking order history got swept in. The override was as narrow as the log evidence justified, and the replay is what proved that, not a code review, not a test suite written before the pattern was known.

This matters because the natural failure mode of a fix like this isn't "it doesn't work." It's "it works, but it's broader than the evidence supports, and it silently changes behavior for accounts you never looked at." A replay against real history is the only check that catches that, because it tests the fix against the full population of orders you actually process, not against a handful of test cases someone wrote after the fact based on their assumptions about what the fix should do.

We've written elsewhere about why this kind of replay discipline matters more broadly, not just for routing but for any automated decision layered on top of operational data (https://aiterated.com/chronicles/who-checks-the-checker-ai-agents-escape-rate). The short version: a fix that looks clean at the point of deployment can still misbehave once it's live, and the only way to know the difference ahead of time is to test it against the actual population it's going to touch, not a sample you picked because it seemed representative.

Where this generalizes beyond routing

The reseller misroute is one instance of a pattern we see constantly in operations-heavy businesses: the org's real logic is distributed across the judgment of the people running it, not centralized in a document anyone can hand you. That's true of routing. It's also true of order entry, ship-date estimation, change-request triage, almost every workflow with enough history behind it to have accumulated exceptions.

The instinct when automating any of this is to build the general case first and patch exceptions later as they surface. That instinct is backwards for any account or order type with enough volume and enough history to have generated real approval-log evidence. If the pattern is there, in the log, consistent across enough orders to rule out noise, it should be the default for that account from day one, not a patch applied after the automation gets it wrong in production and someone on the floor has to notice.

The broader argument for building this kind of software as custom, modeled on your actual operation instead of configured against a generic workflow, is one we make across most of what we build (https://aiterated.com/chronicles/business-logic-chains-explained). Fleet-wide defaults are what you get from generic automation platforms and off-the-shelf routing rules, because generic tools are built to model the average business, not yours. Modeling your actual business logic chains, including the ones only your senior staff can execute without thinking, is what a custom build gets you that a generic platform doesn't.

If you'd rather deploy this kind of change incrementally instead of betting a full cutover on getting every account right on day one, that's the case for modular implementation and parallel operation, which we've covered in more depth here (https://aiterated.com/chronicles/parallel-operation-deployment). Ship the account-level override for the accounts your log supports, run it alongside the existing process, and expand only as the replay evidence tells you it's safe. That's a slower rollout than flipping on a fleet-wide default everywhere at once. It's also the difference between finding your reseller's exception in a replay before launch, and finding it on the floor days after.

One place we could be wrong: if your operation has no account history, a new line of business or a fresh customer base with no approval log behind it, there's no pattern to mine yet, and a fleet-wide default is the only place to start. That's a real condition, and it changes the answer. But if you have years of approval decisions sitting in a log somewhere, the pattern is already there. The only question is whether you look before you ship or after the floor tells you.

If you're routing orders on a fleet-wide default and you've got an approval history longer than a few months, that history is the spec you haven't read yet.

Related: Parallel Operation Deployment: Zero-Disruption Automation · Risk Mitigation Through Modular Software Implementation

Book a discovery call and we'll walk through what mining it actually looks like for your operation.