The pilot reviewer was your eval set

In the quarter before one company pushed its support agent to every channel, the pilot dashboard showed 94 percent correct handling. Two months after the rollout, complaints started arriving from a category the pilot had never seen: a customer changing a delivery address and asking for a split shipment in the same message. During the pilot, an operations specialist sitting next to the test queue had fixed those cases by hand. Nobody logged that they happened. Nobody knew they arrived dozens of times a week. A meaningful share of that 94 percent was the specialist scoring, not the system scoring, and the number was booked against the system.

Who computed the 94 percent

Pilot accuracy has a denominator that rarely gets questioned. Interactions the reviewer corrected on the spot either never entered the count, or entered it as passed-after-human-assistance. Both conventions are defensible, and they measure different things: the first measures how often the system answered correctly on its own, the second how often the human and the system together did. Reports rarely separate the two, so a score about the pair gets carried into a scaling decision as a score about the system.

That is why the week a pilot ends is the riskiest week. While the reviewer is present, mistakes are stopped before anyone sees them, so the pilot looks clean. Take the person out and the same mistakes still happen; this time nobody catches them. The quality curve did not fall off a cliff at scale. It always had that shape. A layer of people was standing in front of it.

We call this the reviewer dividend. Human review during a pilot contributes to production quality and to the management report at the same time, but it only shows up in the second. Removing it at scale removes the inspection line and the inspection record together, and nothing on the dashboard marks either one as deleted.

Four surveys, four definitions of scale

Is there external evidence for this? The 2026 surveys disagree loudly on numbers, and the disagreement resolves as soon as you read the definitions. WRITER surveyed 2,400 executives and employees and found 97 percent of companies had deployed agents in the past year. Caylent commissioned Censuswide to survey 200 senior leaders at organizations with more than a thousand employees in August 2026, and 59.5 percent said they already run agents autonomously in production. S&P Global Market Intelligence puts genuine scale at roughly 11 percent, while Forrester says about 88 percent of pilots never graduate to production at all. Between them sits a March 2026 survey of 650 enterprise technology leaders: 78 percent had at least one pilot in a controlled environment, 14 percent had scaled an agent to organization-wide use.

None of these contradict each other, because they count different steps on the same staircase. Having deployed, running in production, and running at company scale are three separate states. The first is close to universal. The last stays in the low teens. One company can honestly say it runs agents in production and has not scaled any of them.

The widest pair is the one worth staring at: 59.5 percent autonomous in production against 11 percent at scale. If both are accurate, most production agents occupy a narrow duty with a narrow input distribution. That inference matters, because scaling is precisely the act of handling the full distribution, and the tail of that distribution is what the reviewer used to absorb. The step from 59.5 to something past 11 is therefore not a model upgrade. It is tail capacity.

Three jobs the reviewer did, one of them automated

To see the mechanism, look at what the reviewer actually does all day. First, correction: fixing the reply that came out wrong. Second, definition: deciding what counts as wrong, which is an argument between two people about a borderline case, and the argument is where the standard gets written down. Third, absorption: handling the case the system has no coverage for at all, after which the case simply disappears from view. Automation at scale took over the first job, through rules and validators. Nobody took the second. The third was deleted.

The damage comes in two layers. On quality, the tail is exposed again. On management, something worse happens: if nobody is defining what counts as an error any more, nobody is finding errors. The error rate improves, not because it fell, but because the act of measuring it stopped. A Mavenagi analysis published in April 2026 describes the same pattern. Quality in a pilot, it notes, is often sustained through supervision; when a human reviews every interaction, problems are caught early, and at scale that supervision disappears, so an agent that was never evaluated systematically across the full input distribution degrades in ways that stay invisible until customers complain. That piece is vendor-authored and it relays a third-party March 2026 survey, so we treat the two separately: the observation is the vendor’s, the sample is the survey’s.

The same analysis cites Gartner’s call that more than 40 percent of agentic AI projects will be cancelled by the end of 2027, not because the technology fails but because business value is unclear, costs escalate and risk controls are inadequate. Two of those three causes trace directly back to the visibility lost when review is removed. If you cannot see failures, you cannot cost them, and you cannot explain them to risk.

Freezing review into a replayable sample

The answer is not to keep the reviewer. That forfeits scale. The answer is to convert his daily judgments into an asset: every human correction is a labelled sample. Each sample needs five fields, and dropping any one of them makes the sample useless within six months. The original input, the model output at the time, the version the human finally used, the type of correction, and the business scenario it occurred in. The first three are content and answer. The fourth is case law. The fifth is distribution.

With those samples, three engineering moves follow. First, stratify by correction type rather than by business line. Factual error, judgement error, unauthorized action and format violation have different distributions and different fixes, so blending them into one pass rate measures nothing. Second, in the four weeks before scaling, replay the set weekly and read the pass rate rather than the sample count. Third, set an explicit gate: pass rate within two points of the pilot’s human spot-check rate, and no new failure types for two consecutive weeks, before traffic moves up one notch.

Two points is our own threshold, not an industry standard. Its value is procedural: it turns scaling from a meeting decision into a measurement result. A stricter variant is simpler still, and we prefer it for high-consequence workflows: any new failure type must become a sample before any traffic increase is discussed. That is slower, and it converts every incident into added coverage.

One precondition gets skipped constantly. The regression set has to grow out of real correction logs, not out of a design document. A design document lists the failures you imagined. A correction log lists the failures that happened. The overlap is usually small. This also means the first version cannot be outsourced to an outside adviser. It has to come from the person who sat in the review seat, or from the logs he left behind. There is a window here: once the team disperses and the logs are cleaned up, the asset is gone.

Supervision ratio warns earlier than accuracy

A regression set is a slow signal, replayed weekly. For daily reading you need a second number: the share of interactions a human touches, out of all interactions. We call it the supervision ratio. During a pilot it is high, because everything gets read. After scaling it drops to single digits, and that drop is the design intent rather than a problem. The problem is that nobody notices when it drops too fast.

Here is a pattern worth checking against your own dashboard. A supervision ratio falls from 30 percent to 4 percent over two weeks while accuracy holds steady. Most teams read that as good news: people are needed less often. A second reading is equally available and more common: the remaining interactions have nobody able to judge them, or nobody assigned to look. The two situations draw an identical line. The difference is whether the regression pass rate moved with it. Ratio down, pass rate flat, probably good news. Both down, or the set not running at all, means you are holding no evidence either way.

Put the two numbers side by side instead of reporting accuracy on its own. Accuracy is a lagging indicator: it describes the inputs that were processed in the last two weeks. The supervision ratio is a structural indicator: it describes whether you still have the capacity to find problems. The first tells you what happened. The second tells you whether you will be able to see what happens next.

When this is the wrong instrument

Four situations make the regression set cost more than it returns, or make it irrelevant. One, the cost of a single error is low and reversible, internal knowledge search being the obvious case; sampling suffices, and a weekly replay is overhead. Two, reviewers disagree with each other, three people three verdicts; unify the standard first, because otherwise you are collecting noise and a replay will fossilize it. Three, the input distribution is moving fast, as during a product launch; use a rolling window that swaps a portion of samples weekly instead of guarding a historical set. Four, the tail is low-frequency and irreversible, such as fund transfers or permission changes; those should not depend on an evaluation set at all but on hard constraints and an approval path, so the action never gets auto-approved in the first place. That is workflow design, not model capability.

One more case deserves its own line: the reviewer is not reliable either. Tired people approve visibly wrong output, which is common on overnight batches. A human correction log is therefore a candidate label, not ground truth. Turning it into a standard requires someone to re-check a slice afterwards, typically 5 to 10 percent, which is enough to reveal batch-level bias. Skip that step and you have promoted one person’s judgment to the company’s judgment.

Our judgment

Our judgment is that the bottleneck on agent scale is not the model. It is where the eval set comes from. The scarce asset at the end of a pilot is not the orchestration code and not the prompts; it is the several thousand judgments the reviewer made, and most teams close that record along with the ticket when the pilot ends. By the time someone asks for proof of quality, a single accuracy report remains, and its denominator was never explained.

Three moves follow, all of them executable this week. Export the last 90 days of human corrections and sort them by correction type, starting with how many types exist at all; if you cannot produce a single type, the logging gap is the first hole to fix. Then take twenty samples from each of the top three types, build a fixed set under a hundred items, and replay it weekly, recording the pass rate as a curve rather than a number. Then put the supervision ratio and the regression pass rate on one dashboard and gate scaling on the pass rate instead of on the pilot’s accuracy report. The order does not commute: record first, set second, scaling third.

The question we would leave with a buyer: the last time your agent judged something wrong on real input and one of your people corrected it on the spot, when was that, and what type of error was it? If the answer does not come, those judgments left no trace, and the risk has already moved from the people who were watching to the step where nobody is.

References

Digital Applied, AI Agent Scaling Gap March 2026: Pilot to Production (March 2026 survey of 650 enterprise technology leaders: 78 percent with a pilot, 14 percent at organization-wide scale; five root causes explaining 89 percent of scaling failures)

Mavenagi, 78% of Enterprises Have AI Support Pilots, Only 14% Have Made It to Production, 28 April 2026 (vendor analysis; pilot quality sustained by supervision and lost at scale; relays Gartner on more than 40 percent of agentic projects cancelled by end-2027)

First Line Software, The 2026 Enterprise AI Adoption Gap, By the Numbers, updated August 2026 (compiles WRITER 97 percent deployed, Caylent and Censuswide 59.5 percent autonomous in production, S&P Global about 11 percent at scale, Forrester about 88 percent of pilots never graduating)

Caylent, 98% of enterprise leaders would let AI agents run production under the right conditions, August 2026 (Censuswide fielded, 200 senior leaders at 1,000-plus-employee organizations)

Forrester, The State of Agentic AI in 2026: Companies Are Chasing, Few Are Catching (about 88 percent of pilots never reach production)