Agent cost is re-deciding every step

A workflow that runs thousands of times, and thinks from scratch each time

A ticket arrives. An agent reads it, classifies it, pulls the customer’s history, decides whether to escalate, writes a reply. Same procedure every time, different input. The SOP already exists in writing. What does not exist is a path that runs it without asking a model, at every step, what to do next.

That is the default shape of enterprise automation in 2026. On one production business-process platform, customers created 5,115 automations between April and June 2026. Of those, 91.8% were agents that plan at run time. Yet seven of the ten most frequently executed automations were fixed workflows.

Read those two sentences together and the interesting part is not the 91.8%. It is that customers already understood, at the top of the usage curve, that high-volume work belongs in something deterministic. The 91.8% is what happens everywhere else: writing a natural-language SOP and letting an agent execute it is far less work than drawing branches in a workflow editor. The bill for that convenience arrives at run time, on every run, forever.

Where the time actually goes

For agents that use three to five tools, choosing the next step consumed 37% to 43% of execution time. That is time, before token cost. A 170-second task spends 60-odd seconds deciding which tool to call and where its arguments come from.

Two separate cost lines get conflated here. Model price per token is falling fast — Anthropic’s average price per million tokens went from $2.65 in May 2026 to $0.62 in October, a 77% drop in five months. But an agent re-sends its accumulated context on every decision, so the price cut is absorbed by the number of decisions. The bill is roughly decisions-per-task × tokens-per-decision × price-per-token. The first two terms come from your architecture. Only the third is negotiable with a vendor.

Procurement tables usually list the third. That number is real, and it is also the only term a buyer cannot change by choosing differently.

What compiling a procedure actually means

The ReTrace paper is worth understanding structurally, because the method defines the boundary of where it works. Agents write a reason alongside each tool call: “customer asked for a refund, order already shipped, so escalate”. Tool calls alone do not carry that. The reason is where the decision condition lives.

Three stages follow. First, extract the condition, the tool and the argument sources behind each step from traces that include reasoning. Second, merge those extractions across runs into a compact rule document where every rule links back to the runs that produced it. Third, generate an executable workflow from the rule document — and only this last stage is platform-specific.

That is why the same pipeline produces Python for public benchmarks and typed workflow graphs for a production platform. For an enterprise the practical consequence is boring but useful: the expensive part, the rule document, is portable. It survives a change of orchestration platform. A rebuild of the last mile is cheap; a rebuild of the extraction is not.

Production results: what fell and what rose

On 79 production procedures, compiled workflows reached 93.8% completion against 96.8% for the agent, three points lower. Cost per task fell from $0.127 to $0.096, 24% lower, counting model calls made inside tools. Median time fell from 68 seconds to 37.

Mean time rose, from 153 seconds to 163. That number deserves a paragraph of its own. Twenty of the 79 procedures use web automation. Excluding them, compiled workflows average 58 seconds against the agent’s 92, at 19% of its cost. The overall mean is dragged upward by the web group.

Why? A successful web-tool call takes a median 270 seconds. A failed one takes a few seconds. Some of the agent’s apparently fast runs are fast because its calls failed. The paper states the success rates plainly: 86.6% for compiled workflows, 27.3% for the agent. The platform also marks some agent runs as complete with no successful tool call at all.

Which produces a procurement rule you can apply tomorrow: do not compare vendors on completion rate or mean latency. Ask for tool-level call logs. Completion is the platform’s own verdict. Tool outcomes are an external fact. Two vendors can both report 96% while one of them fails a third of its external calls and gets the run marked complete anyway. On a slide, those two numbers look identical.

Latency as a different product

On τ²-Bench the comparison is sharper. Compiled workflows reached 75.8% task success against 80.6% for the agent — 4.8 points lower — at 1.3 seconds per task instead of 170. On SOP-Bench they run in 0.40 seconds against 46.6.

A gap of that size is not an optimisation inside one architecture. It is the difference between asking a model at every step and not asking at all. An agent pays a full model call per branch decision; a workflow pays only where a tool itself needs a model, such as classifying a document. 170 seconds versus 1.3 seconds is not a faster version of the same product. For anything with a human waiting — live agent assist, real-time triage — it is a different product.

One SOP-Bench detail is easy to misread. Unrepaired, compiled workflows land at 80.6%, below few-shot generation at 83.6%. With three rounds of repair driven by labelled training failures, they reach 87.8% against the agent’s 85.5%. The repair signal is “this input failed, here is the correct answer”. Teams without labels cannot reproduce that gain, so the repaired number should not be quoted as a general result.

What reasoning is worth, by procedure length

This is the experiment to remember. Removing the agent’s reasoning costs 8.8 points of enterprise completion and 6.2 points on τ²-Bench. On SOP-Bench it gains 1.7.

The same intervention helps on long procedures and hurts on short ones. SOP-Bench procedures are short with few branches, and their conditions are largely written into the SOP text. Reasoning adds no information there; it restates what the document already says. Enterprise procedures carry tacit conditions — whether order status or shipping status dominates, how refund amount combines with customer tier — that the SOP does not spell out and that a human absorbs from experience. Reasoning is where those get applied.

So “should we compile this?” is the wrong question. The useful question is per segment: within one SOP, which decisions have enumerable conditions, and which depend on judgement. Compile the first, leave the second to the model. ReTrace itself only writes rules for conditions it can read out of the reasoning; it does not invent the ones it cannot.

Payback: roughly 75 runs

Construction is not free. Building one workflow takes at least 11 model calls; an agent’s run averages 12.1 planning calls. Construction therefore costs about eleven runs, and the savings accrue per run. From measured savings and a τ²-Bench construction cost, the paper estimates construction is recovered after about 75 executions.

75 is a low bar. The more useful number is what follows: those seven most-executed workflows each ran 297k to 528k times in three months. At a 75-run payback line, they cleared it within the first hours.

The decision order that falls out is: sort procedures by execution count, then consider compilation. Do not sort by complexity. Complex procedures tend to run rarely, and rarely-run procedures never pay back a build.

A question worth putting to any vendor quoting you a compilation fee: how many runs until this pays back? A quote that cannot answer that is selling a one-time engineering investment as an ongoing service.

Four tests that turn this into a decision

Volume. Under a few hundred runs a year, compiling does not pay — construction equals about eleven runs, plus repair and maintenance forever after. A usable line: annual runs × savings per task should exceed five times construction cost.

Can the condition be written down. The test is whether traces contain reasoning. If the logs hold only tool calls, any recovered rule is a guess, and it fails quietly — no error, just a wrong branch on some class of input. Those procedures need logging before compilation.

Stability of SOP and tools. The paper is explicit that a change to an SOP or its tools may require a rebuild. Procedures that change within a quarter are poor candidates, which excludes many of the ones most eager to save money.

Cost of failure. Completion dropped three points in production. What those three points are worth depends on which ones they are. Sample the failures and classify them; the aggregate completion rate will not tell you whether an unacceptable failure type is in there.

Four categories should not be compiled at all: procedures whose external interaction is non-deterministic (the web group, where mean time actually rose), procedures with many exception branches and no historical traces, procedures whose SOP text is still being edited, and procedures whose failure cost is unacceptable. These are not “later”. They are “fix the prerequisite first”.

What the paper itself concedes

Any summary should carry this section, or readers will file it under product literature.

The authors list five limitations. The reasoning may be wrong, or may not reflect why the agent acted; construction checks confirm the generated code loads and contains no planning calls, and where re-execution is available they count how many logged steps it reproduces, but they do not show that the recovered conditions are correct. Evaluation covers new inputs to the same procedures, not transfer to new procedures. A change to an SOP or its tools may require a rebuild. Poor or incomplete traces limit what can be recovered. And enterprise completion is reported by the platform, which can mark failed runs as complete.

The last one carries the most weight. The 93.8% and 96.8% above are platform-reported, not externally audited. The same reason that makes “ask for tool logs” good advice also applies to this paper’s own headline numbers: treat the three-point gap as a direction, not a measurement.

Our judgement

First, enterprise automation is splitting into two layers that will coexist rather than compete: compiled deterministic workflows for high-volume procedures with enumerable conditions, and run-time planning agents for low-volume judgement-heavy procedures whose SOPs are still evolving. The problem today is not choosing between them. It is that 91.8% of new builds land in the second layer while seven of the ten most-executed automations live in the first. That mismatch is the addressable cost.

Second, cheaper models will not fix the mismatch. A lower price per token offsets token cost. It does not offset the number of decisions per task, or the seconds each decision takes. The metric to watch is decisions per completed task, not dollars per million tokens.

Third, one change is available immediately: take completion rate and mean latency out of your vendor comparison and replace them with tool call success rate, decisions per task, and full cost per accepted result. Vendors define the first two themselves. They cannot define the third.

Three things to do this quarter

One. Sort your automation inventory by executions over the last three months. Take the top twenty. Mark each as agent or fixed workflow. For every agent still in the highest-volume group, estimate how much of a single task goes into deciding the next step — use the 37-43% band as an upper bound. That is enough to rank the work.

Two. Pick three procedures for a compilation pilot and write the acceptance test against tool call outcomes, not completion rate. Six weeks, one output: cost per task, decisions per task, tool success rate. Those three numbers are the basis for every negotiation that follows.

Three. Walk the SOP and mark which conditions you cannot write down. Do not force those into a workflow. Add logging first, so the next run leaves a reason behind. This step costs nothing and determines whether the next compile can be done correctly.

Sources

- ReTrace: Distilling Tool-Using Agents into Planner-Free Workflows, arXiv:2611.14492v1, submitted 10 November 2026: arxiv.org/abs/2611.14492v1- All figures above — 5,115 automations, 91.8% agents, seven of the top ten fixed workflows, 37-43%, 93.8% vs 96.8%, $0.127 to $0.096 (−24%), median 68s to 37s, mean 153s to 163s, the 20 web procedures at 58s vs 92s and 19% of cost, 86.6% vs 27.3% web success, median 270s, τ²-Bench 75.8% vs 80.6% at 1.3s vs 170s, SOP-Bench 80.6%/83.6%/87.8%/85.5% and 0.40s vs 46.6s, the −8.8/−6.2/+1.7 reasoning ablation, at least 11 construction calls and 12.1 planning calls per run, ~75-run payback, and the seven workflows at 297k-528k runs in three months — come from the paper’s main text and appendices. - The limitations section maps one-to-one onto Section 7 of the paper. - Anthropic per-million-token pricing (May to October 2026) is cited from Tom’s Hardware reporting on The Register’s figures, used only as context for why cheaper tokens do not close this gap: www.tomshardware.com/tech-industry/artificial-intelligence/firm-that-lets-you-rent-out-your-idle-mac-for-ai-inference-workloads-warns-of-plummeting-token-costs-darkbloom-network-sees-earnings-drop-to-usd1-23-a-day-as-anthropic-tokens-fall-77-percent