Nobody Owns the Exceptions

Here is a failure that looks nothing like a failure while it is happening.

A business automates its invoice intake. The system reads the documents, pulls the values, files them. It works. In testing it took ninety-two percent of a real sample all the way through and set the remaining eight percent aside for a person, which everyone agrees is a good result, and it goes live in March.

In April the bookkeeper mentions that some invoices are ending up in a folder. In May the folder has four hundred items in it. By June, when someone finally looks, the folder contains a mix of genuinely odd documents, a few that the system got wrong, and a large number that were fine but got deferred because of a supplier who changed their template in April. Nobody was told to check the folder. Nobody was given time to. The bookkeeper has quietly gone back to processing everything by hand, because doing half the work in one system and half in another was worse than doing all of it one way.

The AI was not the problem. The ninety-two percent still works. What killed it was the eight percent, and specifically the fact that the eight percent was treated as a residue instead of as a job.

Ten percent is a job, not a rounding error

The arithmetic here is the thing people skip.

Take a process running a thousand items a month. A system that defers eight percent produces eighty items requiring a person. That does not sound like much until you account for what those eighty items are: they are, by construction, the hardest ones. The straightforward cases got handled. What is left is the ambiguous document, the record that matches two customers, the value that makes no sense, the case nobody anticipated.

Exceptions take longer per item than routine work, because they require understanding rather than processing. Put a number on it: if a routine item took two minutes by hand and an exception takes eight, those eighty items are about eleven hours a month. At a loaded $40 an hour that is $440 a month, or $5,280 a year, arriving unevenly and clustering around whatever caused the anomaly.

That number belongs in the business case. When the payback arithmetic counts the hours saved on the automated ninety percent and quietly assumes the remaining ten percent costs nothing, the case is wrong by a margin that often decides whether the project was worth doing. A build that saves sixty hours a month and creates eleven hours of exception work saves forty-nine, and forty-nine is the honest number to put in front of anyone approving the spend. On those figures it is still a good project. The point is that it is a different project from the sixty-hour one in the proposal.

What actually lands in the queue

It helps to be specific, because the categories behave differently and want different treatment.

Structural surprises. A document in a format the system has not seen, a field that moved, a supplier who redesigned their template. These arrive in bursts, all from one source, and they are the cheapest to fix permanently because one change removes the whole cluster.

Genuine ambiguity. Two customer records that could plausibly be the same person. A line item that could be one of two categories. These are irreducible: the information needed to decide is not present in the document, and no amount of engineering produces it. A person has to know something the paperwork does not say.

Out-of-range values. A quantity, a price, a date that falls outside anything plausible. Usually a data entry error at the source, occasionally a genuinely unusual transaction. Cheap to detect and worth detecting, because these are the ones that cause real damage if they pass.

Low-confidence outputs. The system produced an answer and does not trust it. This is the category a well-placed checkpoint exists to catch, and it should be the largest share of a healthy queue.

Downstream failures. The system did its part and the next system refused it: a record locked, an API down, a validation rule in the accounting package. These are not AI failures at all, and they are the category most likely to be silently dropped, because they occur after the point where anyone is watching.

Knowing which of the five you are looking at changes what you do about it. The first and third are engineering problems with permanent fixes. The second is a permanent cost of doing business. The fourth is a threshold to tune. The fifth is an integration problem wearing an AI costume.

Designing the queue

The queue needs six things, and none of them is technically difficult. They are simply the parts that get left out when the attention is on the automated path.

A named owner. One person, by name, with a named backup for holidays and busy weeks. In a business of fifteen this is usually somebody already doing another job, which is fine, provided the time is acknowledged instead of assumed to be free. It usually lands on the owner, who according to CFIB is already working 54 hours a week, and 59 where labour shortages bite, which is the honest context for adding eleven hours a month of anything.

A place with a size you can see. The queue has to be visible as a count, not as a folder somebody could open. The invoice folder in the story above was invisible precisely because looking in it required a decision to look. A number on a screen that someone sees daily is the whole difference.

A time limit. An agreed maximum an item can sit unreviewed, and a defined behaviour when it is exceeded. Escalate, alert, hold the process, whatever fits, but decided in advance instead of discovered.

Triage. Not all exceptions are equally urgent. A deferred invoice can wait two days. A deferred customer commitment cannot. Sorting on arrival costs almost nothing and prevents the queue being cleared in the order things happened to land.

A route back in. When the person resolves the item, the result has to rejoin the process automatically. If clearing an exception means re-keying the result into three systems by hand, the queue becomes the reason people stop using the system, which is the outcome the whole build was meant to avoid.

A log. Every exception, its category, and what the person did about it. This is the item most often skipped and the one that determines whether the queue shrinks.

The log is where the value is

An override is not a correction. It is evidence.

When the same field is corrected the same way thirty times in a month, that is not thirty errors. It is one defect in the specification appearing thirty times, and fixing the cause removes the whole class. When one supplier accounts for half the queue, that is a template change, not an AI limitation. When a category of ambiguity keeps recurring, the useful response may be to change the upstream process so the information needed to decide is captured at the point it is known, rather than reconstructed later by someone reading a document.

This is the mechanism by which the exception rate falls, and it is the only one. Nothing about a system makes it improve on its own. A monthly half hour reading the log, asking what caused the top three clusters, and turning the answers into changes is what separates a queue that shrinks from a queue that becomes a permanent tax.

There is a second use for the log that matters more than it sounds. It is the record of what the system got wrong and who caught it. When somebody asks in eleven months how a particular figure was arrived at, or whether a system has been performing as expected, the log is the answer. Businesses that keep one can demonstrate their process. Businesses that do not are relying on memory.

And a third use: the log tells you when the system has quietly got worse. A queue that has run at eight percent for six months and is now at fourteen percent is telling you something changed, whether upstream, in the data, or in the model behind the system. That is the earliest available warning, and it costs nothing to watch.

The part nobody wants to hear

Some exceptions never go away, and a design that assumes otherwise is dishonest.

Genuine ambiguity is permanent. If two customer records are indistinguishable in the data, no engineering resolves it, and a person will be deciding that case for as long as the business runs. The right response is to make that decision fast and well-supported instead of to keep promising it will be automated next quarter.

That permanent share is the real floor of the operating cost, and it deserves to be stated out loud in the estimate. A vendor who says the exception rate will approach zero is either inexperienced or selling. A build that says “roughly five percent of these will always need a person, here is what that costs you annually, here is why it is still worth doing” is one you can plan around.

The part worth carrying out the door

The automated path gets all the attention because it is the part that demonstrates well. The exception queue gets none, and it is where the project is decided.

Before anything is built, the questions are short. What share of cases will come back, and on what evidence. Who owns them, by name, and who covers when that person is away. Where does the queue live, and can anyone see how big it is without opening a folder. How does a resolved item get back into the process. Who reads the log, how often, and what happens to what they find.

A build with good answers to those has thought about the whole job. A build without them has automated the easy ninety percent and handed you a new problem with no name on it. The ninety percent is the demonstration. The ten percent is the business.