Pilot to production: swap points nobody writes

Month five of the pilot, and the demo still runs clean. Twenty users, data cleaned in advance, endpoints pointed at a mirrored environment, one person quietly catching failures by hand. The finance lead asks an ordinary question: if this one transaction is wrong, who finds out, and when. The room goes quiet for a few seconds. Nobody is lying. There is simply no answer yet, because the answer only becomes visible when twenty users turn into two thousand, and by then there is no time to build it.

4.7 months is not how long the model needs

A survey of 650 enterprise technology leaders, fielded in February and March 2026, gives the shape of this wave: 78% of organisations have at least one agent pilot running, and only 14% have scaled an agent into organisation-wide production use. The average pilot lasts 4.7 months before it stalls. Among organisations that tried to expand scope or volume, 64% hit blocking issues, and of those, 72% have been stuck for more than six months with no clear path out.

The sector split carries more information than the headline. Financial services reaches production at 21%, healthcare at 8%, manufacturing and retail between 13% and 16%. The variance tracks two things: how standardised the document-heavy workflows are, and who is willing to sign for the consequence of an error.

The same survey groups the causes into five, which together account for 89% of scaling failures: integration with legacy systems, output quality that becomes unstable at volume, absence of monitoring tooling, unclear organisational ownership, and insufficient domain training data. Those five are not parallel. Unclear ownership leaves the monitoring unfilled, missing monitoring makes quality problems invisible at volume, and by the time they are visible they have compounded. A list of five that is really one chain.

The 4.7 is most often misread as the model needing more time. It is an accounting fact: the date the temporary arrangements come due together.

Between demo and production sits a layer of swaps

Set the two verification targets side by side and the difference is one sentence. A demo verifies that it can be done once. Production verifies that every run can be checked by someone else. What sits in between is eight specific swap points. None of them appear in the demo script, because the demo passes without them.

One, data. A snapshot becomes live data, and the distribution has to match. Demo data was cleaned: missing fields, dirty values and changed definitions were all handled before you saw them. Live data has no obligation to cooperate. Quality variance does not grow linearly with volume, it grows with surface area.

Two, endpoints. A mock becomes a real endpoint with real permissions and real rate limits. That swap changes the shape of every error: instead of returning plausible fake data, it now returns timeouts, 403s, 429s, missing fields and values that contradict the documentation. Error handling written during the demo is usually rewritten here.

Three, review. One reviewer becomes a queue. A person can read twenty transactions a day with attention; a queue holds two thousand. Review changes shape from reading everything to reading exceptions, which means someone has to define what an exception is, set the threshold, and decide what happens when the threshold is wrong. Skip this step and you have put one person in a flood.

Four, latency. Interactive latency becomes a service level. In a demo somebody is waiting, so slow is tolerable. At volume most requests have nobody watching, and anything that does not return gets retried three times.

Five, retries, the one most often skipped. Manual retry becomes idempotency. Retries almost never happen in a demo and start in week one of volume. A non-idempotent retry pays twice, emails twice, opens the same ticket twice. Fixing it touches the data model rather than the code, which is why it slides to the next phase.

Six, evaluation. Sampled evaluation becomes full instrumentation plus scored sampling. Sampling can tell you this class is acceptable on average. It cannot see the tail, and the tail is where complaints come from, because complaints are always about one specific transaction.

Seven, credentials. Temporary credentials become a revocable identity, and that identity has to answer three questions: who issued it, when does it expire, and does a downstream delegation expire when the upstream authority does. The third question is the one usually missing and the one an auditor asks first.

Eight, logs. Locally written logs become externally retained logs, with a retention window no shorter than the one on the business record. Where the two windows differ, the interval between them is the period in which you have a control that you can describe but cannot evidence.

Why the swaps always slide to month 4.7

Because they belong to nobody. The model team delivers a model, the business line delivers a process, IT delivers an integration. Swap points fall in the seam between the three: doing them early earns no credit, doing them late is how you get noticed.

Delay also has a reasonable-sounding case. Remove any one swap point and the demo still passes. Remove half of them and the pilot still runs. That property, being invisible to the current metric, makes them the easiest work to schedule for the next phase. By the fourth postponement the user count has grown tenfold.

Writing them down costs a page. Not writing them down costs a due date, and that date usually arrives without a budget line attached. Of the five failure causes, unclear ownership is listed fourth, yet it amplifies the other four, because the repair for each of them needs a signature.

The seat nobody sits in

The most operational finding in that survey is the structural practice shared by teams that crossed the gap: they created a dedicated AI operations function, separate from both IT and the business unit, owning the evaluation framework, production monitoring and incident response. Teams that left the responsibility diffused across existing functions consistently failed to scale in the sample.

That function has to fill three seats, and the three cannot be the same person. Who approves the pipeline to touch real business. Who can stop it without asking. Who writes down the basis for each transaction. When the three collapse into one, the accounting is what gets sacrificed, because it produces nothing at the moment it happens.

None of this is a procurement question. Buying a tool does not create a seat; a tool only gives the person in the seat something to look at. Which means the question for a vendor is not how many scenarios the platform supports. It is who can stop it after go-live, and how long that takes to take effect.

Narrow survives before broad does

The second shared trait among the successes: they all started with a single-function, narrow task, a document classifier, a data enrichment pipeline, a routing agent. Scope expanded only after the narrow version had been stable, never designed in from the start.

The mechanism is testability, not caution. A narrow task has an input space you can enumerate, which closes the evaluation set and lets you answer whether this one transaction was right. A broad task has no enumerable input space, so evaluation falls back to sampling, and sampling can only say the class looks fine on average. Quality variance compounds with volume and edge cases never finish being tested, so every expansion multiplies errors you cannot see.

Which is why the gate for expansion should be written down: ninety consecutive days without a severe incident, full instrumentation in place, and only then the next tier of scope. Ninety days is not ceremony. It has to cover a month end, a quarterly close and at least one upstream API change.

What this data does not prove

Sources first. The 78%, 14%, 4.7 months, 64% and 72% all come from one survey run in February and March 2026 and republished by a media outlet. They are self-reported by respondents. The 4.7 is a mean, not a promise, and the distribution across sectors and processes is wider than the mean suggests.

The 89% is respondents attributing failure causes, not a causal decomposition. Unclear ownership appearing alongside scaling failure can be read as ownership causing the failure, or as both coming from an earlier decision, such as not naming an owner when the project was chartered.

We have no client sample to cross-check against, so the position here comes from an independent rearrangement of public surveys and documented failure patterns, not from any delivery record. Read each number by its source type; survey results, means and attributed proportions are not interchangeable.

Our judgement

The gap between pilot and production is not a size gap, it is a gap in what gets verified. The moment volume arrives, the object of verification changes from one demo to the evidence behind every transaction, and the eight swap points are the gates on that line. Writing them down costs a page. Not writing them down costs four months, and those four months never appear on anyone’s account.

Three things follow that can be done immediately. First, attach a one-page swap point list to the project charter, with three columns per point: who owns it, when it gets swapped, how the swap gets verified. The list belongs on page one rather than in an appendix, because it is the only thing that surfaces the due date in advance.

Second, put the three seats into the process with names attached: approve, stop, and account. Write the stop latency into the service level. This is cheaper than any tool selection and more effective than any tool selection.

Third, make the expansion gate testable: ninety consecutive days without a severe incident, full instrumentation coverage, and every swap point closed. All three, or no expansion. Two out of three is a no.

The question to leave with buyers: if you moved the pilot from twenty users to two thousand tomorrow, which single thing would break first? If you can name it, that is the swap point to write down today. If you cannot, then four months from now someone in a review will ask that ordinary question again.

References

Digital Applied, “AI Agent Scaling Gap March 2026: Pilot to Production”, 26 March 2026 (survey of 650 enterprise technology leaders: 78% with pilots, 14% at production scale, 4.7 months average stall, 64% hitting expansion blocks with 72% of those unresolved past six months, five root causes at 89%, dedicated AI operations function and narrow-scope starting point)

KG Solutions, “Multi-Agent AI Orchestration: A CTO’s 2026 Guide” (starting from one high-value workflow with two or three agents, and the phased timeline; vendor article)

Domino Data Lab, “Enterprise design patterns for multi-agent AI workflows”, 1 October 2026 (its 2026 Enterprise AI Report surveyed 639 senior AI leaders at $100M+ enterprises; the correlation between integrated governance and running agentic AI in production is cited from that piece; vendor article)

The swap point list, the three seats and the ninety-day expansion gate are OOMeta’s own synthesis of the public material above, not a finding published by any of these organisations.