Where agents stop: 5M signals to 5 alerts

A company spent 2 years wrapping another company’s open source gateway. Each customer wanted its own keys, routes and limits, so the wrapper grew. By the time it passed 100,000 lines, a routine upstream upgrade broke connections. They rolled back. They never found out why, and they have not upgraded the upstream directly since. That is what a lost boundary looks like from the inside. We see it more often than any feature list suggests.

What the wrapper bought, and what it cost

Respan told this story in October 2026, when it launched a Rust gateway on Hacker News. The old approach worked commercially. Thousands of organisations were served, and every one of them wanted something slightly different.

So the wrapper kept growing. Then one January upgrade produced intermittent connection errors, with no diagnosable cause.

The line count is not the interesting part, because the debt did not make the system slow. It made the system unable to move. Any upstream change became a risk assessment, and any bug had to be traced through 100,000 lines of the company’s own code before the upstream could be suspected.

Their conclusion was blunt: patching the upstream was not sustainable. Their new design answers with structure. The core dispatches events and nothing else. A provider is a short file, 22 lines for Mistral, and each quirk gets its own test. Extensibility stopped being a promise and became a property of the layout.

There is a second signal in the same market. Palo Alto Networks announced its intent to buy Portkey on 30 April 2026 and closed on 29 May. Mintlify acquired Helicone on 3 March 2026, and Helicone moved into maintenance mode.

The control plane that used to be a product category is being absorbed into security platforms, which means selling a gateway on provider coverage is a shrinking business. What is left is the boundary inside your own system. If you run a gateway today, the test is simple: can your team upgrade the upstream inside one release cycle?

Two jobs for one model

CoreWeave published the opposite case on 15 October 2026, from its own data centres. Power draw, inlet temperature, coolant flow and fan speed are sampled continuously. That adds up to about 5 million signals a day, and your operations team cannot read them either.

So the company split the work in two. Detection runs as a deterministic pipeline that agents assembled offline. Explanation runs as an agent inside the live system, and it only handles the 5 or so events a day that the detector surfaces.

The split follows from the requirements. Finding an event repeats millions of times, so it must give the same answer every time. Explaining it has no procedure that can be written in advance, because the useful procedure is the one you write after you have seen the case.

Put differently, an agent can build the tool, or it can be the tool. One question tells you which happened: when the system runs, is the agent still there?

If it built the pipeline and left, production runs code and a fixed model. The same input always gives the same output. If it is the tool, it is called per event, and the same input may not.

The cost curves run in opposite directions too. Building the tool is expensive once and cheap per data point afterwards. Being the tool is cheap to start and charges on every call, with latency that grows alongside.

Most teams pick the second, which is why a demo lands in a quarter. That means the bill arrives at scale: 5 million signals a day, one inference each, and the queue time multiplies.

Model drift sits on the same ledger. When the agent is the tool, behaviour moves with the model version, so every new backbone needs revalidation. When the agent built the tool, drift stops at handover. What runs afterwards is code that diffs cleanly.

The harness holds the design, not the prompt

What CoreWeave gave the agent was not a prompt but a working environment. It had tools to profile telemetry, engineer features, train candidates and score itself against a fixed held-out set of labelled incidents.

It could rewrite code. It could not move the scoring target. The loop ran until the metrics stopped improving, and a person reviewed the output before release.

The expensive work sat outside the model. Which signals carry a failure, and how to represent them, came from sitting with the engineers who run the fleet, not from prompt tuning. The definition of a real incident was written by people, and that definition is what the agent scored itself against.

Their own summary is blunt. Point an agent at a complex system with no requirements, and it will detect plenty that means nothing.

The live agent has a different constraint. Asked to explain an anomaly from memory, a model produces something plausible and occasionally invented. This one cannot answer from memory, because every claim has to come from a tool call it actually ran.

That does not make the answer correct. It makes the answer checkable, and checkability is the acceptance criterion here. In practice, that is the difference between a demo you trust and a system you can audit.

One more rule is stated without hedging: the agent does not act. A root-cause judgement can be wrong, and on a live cluster that error costs far more than the automation saves. So every recommendation reaches a person with the evidence attached.

Precision is a design requirement, not a tuning target. A detector that raises 1,000 alerts a day is worse than useless. Operators stop reading it, and the real event lands in a queue nobody trusts. Letting the reasoning agent see only what the detector surfaced keeps cost bounded, and it keeps attention on the events worth reading.

A variable process that hands over a fixed artifact

The result reads as a contradiction. A non-deterministic process produced a deterministic artifact.

The agent varies between runs. What it delivers does not. Feature definitions, model weights, thresholds and post-processing are ordinary code plus a fixed model. They are versioned, diffable, and read by a person before release. Run the detector twice on one signal and you get the same answer, 5 million times a day.

The line moves over time, and that is the part most deployments miss. On the reasoning side, an agent saw the same class of anomaly twice and proposed a deterministic rule for it.

That proposal went to the engineer who owns the hardware. He is the one who can judge whether the rule describes a failure worth alerting on. An agent used as a tool ended up building a tool.

Their summary of the economics is the line to keep. Most agent deployments get more expensive as they scale, and this one is built to get cheaper. Work starts expensive and flexible, and whatever proves repeatable gets codified and moved to the cheap side.

The move is approved by a person, one cell at a time. It is not triggered by a better model arriving, and that is the part we would put in front of a buyer.

66.7% of allocations from code marked off

The third case has nothing to do with agents. It closes the other half of the argument, because the gap between declared and actual behaviour can only be closed by measurement.

Cloudflare shipped on-demand CPU and memory profiling for Workers and Durable Objects on 9 October 2026. It captures a window of production traffic from the dashboard or the command line, renders a flamegraph, and exports pprof. The CPU sampler fires once per millisecond, and the runtime holds an isolate lock only while starting or stopping the profiler.

Their first example is one of their own Workers. It sat at about 133 MB at P999 against a 128 MB limit, so it was evicted repeatedly for exceeding memory. The team knew memory was the problem, and they did not know which code caused it.

A heap profile in pprof showed Prometheus instrumentation taking roughly 66.7% of allocations. That code was supposed to be disabled, and the team believed it was.

It was half disabled. The instrumentation still hung off many code paths, and the data it collected never left the Worker, so the memory cost was paid in full. Removing the path took P50 from 70 MB to 54, P90 from 94 to 79, P99 from 113 to 97, and P999 from 133 to 118. That left about 10 MB of headroom below the limit.

A second example is smaller and just as useful. In the Worker behind the R2 binding, a function called genericR2JsonReplacer took over 5% of CPU. It was called by JSON.stringify on every node, and it walked the tree itself, so a value nested 5 levels deep was processed 5 times. Removing the duplicate work made that function 2.7x faster.

A duplicated metrics call cost another 1% of CPU per invocation, and caching the result removed it.

These numbers belong in a piece about agent engineering, because agent systems sit where conventional monitoring is least sensitive. They stay resident, spike on CPU, and grow on memory in the tail.

A dashboard tells you the number went up. An allocation profile tells you which line is paying. The awkward finding is that off does not mean stopped, which is why we now treat every disablement as unverified until a profile says otherwise. That holds for disabled plugins, commented-out hooks, and any configuration value that claims a feature is gone.

One line, three measurements

The 3 cases describe one thing. The defensible position sits in the engineering layer around the model, and what defends it is not code volume. It is a line that is written down and measured.

That line has 3 segments. What must be deterministic, meaning the judgement on the live path. What may reason, meaning explanation and recommendation. What is an extension, meaning plugins, scripts and configuration. Each segment needs a measurement, or the claim stays a claim.

The deterministic segment is scored against a held-out incident set, with the target fixed and the means open. The reasoning segment is scored by provenance, where each conclusion points back to an execution that happened. The extension segment is scored by an allocation profile taken after a disablement, checking whether P50 and P99 moved.

None of the 3 depends on a model vendor, and you can run all three without asking one for permission. That is why they survive a change of backbone.

A May 2026 preprint makes the same case from the other direction. The authors leave model weights alone and adapt only the runtime harness, evolving one called Life-Harness from training trajectories. Then they freeze it and test on unseen tasks.

They ran 8 environments from AgentBench, τ-bench, τ2-bench and AppWorld, across 18 open-weight backbones. With Qwen3-4B-Instruct, mean accuracy rose 28.7 points over Meta-Harness and 7.5 over AHE, while using 8.5% and 68.4% fewer optimisation tokens.

The same harnesses also helped GPT 5.5, Opus 4.8 and Gemini 3.1 Pro, adding 3.5 points on average. Read that gap as the honest number, because the large gains come from rule-governed benchmarks. This is a preprint reporting its own results, without peer review.

We ran the same check on our own stack. Our runtime is an unmodified git clone of upstream, 0 commits ahead and 0 behind. HEAD sat at 22 October 2026, with 2 untracked files in the working tree. So our fork cost today is zero, and all our capability sits in scripts, plugins and skills. The check is one command, git rev-list --left-right --count HEAD...@{u}, and it should keep returning 0 and 0.

The reason is Respan’s. Once a fork opens, every upstream release has to be re-evaluated, and the day comes when nobody can afford the evaluation.

Where the evidence stops

CoreWeave’s numbers come from its own blog, and the company sells physical AI field engineering. It is an interested party reporting on itself, and 5 million to 5 describes its fleet, not yours.

Respan’s 100,000 lines are 1 company’s history, not an industry statistic, and its gateway is still pre-alpha. Cloudflare’s 66.7% and 2.7x are 2 specific cases, not a distribution. On-demand profiling also needs production traffic, so a low-traffic version yields no sample. The preprint’s gains vary widely by backbone.

What transfers is the mechanism, not the percentages. Keep only deterministic decisions on the live path. Require every reasoning claim to point at a tool call, and keep the agent from acting directly. Keep your upstream upgradeable, and put extension work in plugins. Re-measure after a disablement instead of trusting your configuration.

None of the 4 needs CoreWeave’s data centres, so you can start with what your own system already records.

Our judgement

Our judgement is that the engineering layer around the model is the defensible position. What defends it is 3 measurable lines, not code volume, and our preference is a smaller system we can measure over a larger one we cannot.

Models change, vendors raise prices, and gateways get absorbed by platform companies. All 3 happened in 2026. The question of which part of your system must return the same answer can only be answered by the people who own it.

3 moves follow. First, write a harness specification at kick-off, with fixed targets, open means and evidence requirements. Review the specification, not the prompt.

Second, give every upstream dependency an upgrade budget of at least 1 successful upgrade per release cycle. If you can upgrade, the extension boundary is still yours. If you cannot, the wrapper crossed the line, and our advice is to move capability into plugins rather than add another layer.

Third, treat disabled as unverified in your own environment. After switching something off, capture an allocation profile and record P50 and P99 in the change log. It is the cheapest of the 3 and the easiest to skip, because off sounds finished to your own team.

The question worth leaving with a buyer is this: if you swapped the model tomorrow, which part of your system would behave exactly the same? If you can name it, that part is your position. If you cannot, the position belongs to someone else.

References

CoreWeave Engineering, Agents on live hardware: reason where you must, engineer everywhere else, by Joël Henry, 15 October 2026 (five million signals, the two agent modes, the held-out incident set, the tool-call requirement and the proposed rule all come from this post)

Hacker News, Launch HN: Respan (YC W24), October 2026 (the 100,000-line wrapper, the January upgrade failure and the manual backports are the founder’s own account)

Respan Gateway source (MIT / Apache-2.0, pre-alpha; plugin layout and provider file length per repository)

Cloudflare, Introducing on-demand CPU and memory profiling with flamegraphs for Workers and Durable Objects, 9 October 2026 (133 MB at P999, 66.7% of allocations, the four post-fix memory readings and the 2.7x function speedup)

Palo Alto Networks press release, completion of the Portkey acquisition, 29 May 2026

Tianshi Xu et al., Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents, arXiv:2605.22166 (preprint, self-reported results)