Agent traffic shows agents are still monolithic

A request enters an agent application and goes around several times: it reads a document, calls retrieval, runs a piece of code, then goes back and changes a parameter. From a person’s view that is already a process. From a service-call view it touched one service from beginning to end. After analysing agent telemetry from more than a thousand of its customers, Datadog found that 59 per cent of agentic application requests made a single service call, and only 18 per cent made three or more. The multi-agent architecture on the slide is rare in real traffic.

How those two numbers were produced

Start with the definition, because it sets how far the numbers travel. Datadog counted end-to-end agentic applications, only those with at least one service call, then measured how many services one complete request crossed. The sample comes from customers using its LLM observability product, and the report states plainly that such companies tend to be serious about infrastructure and skew cloud-native, so this is not a census of the market.

Even with that discount, the gap between 59 and 18 is worth reading. If multi-agent had become normal, the heavier share would sit at the three-or-more end. The opposite is true: close to six requests in ten stop at a single service. That does not make multi-agent useless; it says most teams have not reached it in production. What runs in production is a loop inside one process: read, call a tool, look at the result, read again.

The report offers a corroborating detail. Only 18 per cent of requests crossed three or more services, so the other eight in ten were not handing work between services but circling inside one. Loops of that kind do not appear in service topology; they appear in token counts and latency. That is why a service map can look clean while the bill and the response times resist explanation.

The complexity did not vanish, it moved inside one process

Where the complexity comes from. Another Datadog finding is that framework adoption nearly doubled in a year: from a little over 9 per cent of organisations in early 2025 to almost 18 per cent in early 2026, with the number of services using agentic frameworks more than doubling over the same period. Frameworks make common patterns easy to write, and tool fan-out, retries and branching are one import away.

The trouble is that each of those patterns lengthens the loop, and once the loop lengthens, the engineer’s model of the runtime falls behind. In the report’s words, framework boilerplate makes tool fan-out, retries and branching easy to add, cost and latency drift upward, and failures become harder to reproduce. That is not a defect of frameworks; it is their pricing. You save writing time and pay in explainability.

The visibility gap sits exactly there. Service-level monitoring sees which service a call entered and how long it took. It does not see how many times the loop turned inside that service, how much context each turn carried, or at which turn repetition began. Watch both layers as one and the familiar scene follows: the dashboard is green while the invoice and the complaints arrive together.

The failure mode is not a wrong answer, it is a rate limit

Datadog’s failure distribution is a little surprising. In February 2026, 5 per cent of model calls returned an error, and 60 per cent of those errors were exceeded rate limits. By March the overall error rate had fallen to 2 per cent, and rate-limit errors still numbered nearly 8.4 million, about a third of the remainder. The most common production failure is not a bad answer; it is a request that never gets in.

The causal chain is easy to underestimate. An agent that hits a limit retries, retries raise concurrency, concurrency hits the ceiling again, and a local wobble becomes a sustained outage. A monolith is especially exposed here because every step shares one quota: a burst in retrieval stalls every step behind it. Split those steps into services with separate quotas and the propagation path shortens. That is the first real benefit of splitting, and it has nothing to do with agents being smarter.

The remedies are not new: queues, backoff, spare capacity, ceilings on loop length. The report recommends that platforms build budgeting and backpressure into the runtime, and force termination when a loop exceeds its maximum calls or tokens. These are old distributed-systems moves, now applied to a program whose control flow the model decides.

Split or not: three domains, not a maturity badge

Splitting is an isolation question, not a sophistication question. It lands on three domains. Failure domain: if one step breaks, does it drag the others down. Permission domain: do different steps need different data authority, for instance reading tickets versus writing to a payment account, which should not share one identity. Budget domain: do different tasks need their own ceilings so an open-ended task cannot consume the month.

If one of the three must be isolated, a service split has a reason. If none does, the monolith is cheaper, because crossing a boundary has real costs: context must be handed over, traces must be stitched across processes, and tools have to appear in the service map. The report makes the same point about multi-agent systems: debugging them requires propagating context and traces across service boundaries, and managing the platform requires service maps that include tools.

Read 59 per cent against that standard and the conclusion shifts. Most teams have not reached the isolation stage; their complexity sits inside the loop, not between services. Bringing in an orchestration layer now buys topology they cannot use and adds middleware that itself needs observing. The work that pays first is making the single loop visible.

A model portfolio is already orchestration

One clue gets overlooked. Datadog found that more than 70 per cent of organisations run three or more models, and the share running six or more nearly doubled in a year. A model portfolio is an orchestration problem in itself: which step goes to which model, who takes over when one fails, how a retirement is migrated. Its return arrives earlier than multi-agent, because it changes cost and quality directly without changing topology first.

This path has its own traps. The same prompts and tools behave differently across models, so every added model brings another fingerprint of quality, latency and cost, and evaluation load grows with it. The report calls this model churn a governance problem in practice: teams adopt new releases quickly and retire old ones slowly, so they end up maintaining several unrelated paths at once.

So the first layer of orchestration is not agents collaborating, it is routing requests between models: one gateway, one online evaluation loop, a policy per step. All of it works inside a monolith. When a step genuinely needs its own scaling, its own authority or its own ceiling, split it then, and the split becomes a migration with a stated reason rather than architecture theatre.

Four questions for platform selection

Feature lists are easy to be led by. How many collaboration patterns a platform supports, how elaborate a graph it can draw: against 59 per cent those questions carry little weight. Four better ones. Can it leave a trace of every step of one request, including tool results and retries. Can it set ceilings at runtime and stop a runaway loop. Can it assign different models and quotas per step. When a model version retires, is the migration path enumerable.

What those four share is that they point at observability and constraint rather than topology. A platform that answers them can serve a monolith for a long time and still allow a split when one is warranted. A platform that only draws collaboration diagrams keeps adding surface you have to maintain before you reach the stage it was designed for.

None of this makes multi-agent wrong. The 59 per cent is a coordinate in time: most organisations have not arrived at the step that needs it, and the few that have usually got there because permissions or quotas hit a wall first, not because they wanted a better diagram.

Our judgement

Our judgement is that agent orchestration is being discussed as a topology question when it is an observability and isolation question. Real traffic shows most applications are still one process going around in a loop, and their pain is invisible inside that loop rather than disconnected between services. Make the loop visible first, then decide whether to split; reverse the order and you buy capability you cannot use.

Three moves follow. This week, measure one number: what share of your agentic requests make a single service call, and what share make three or more. That ratio is your architectural reality, whatever the purchase list says. Second, break failures down by type, rate limits, timeouts, parse failures, downstream rejections. If rate limits lead, build backpressure and loop ceilings before an orchestration layer. Third, write down three isolation criteria, failure domain, permission domain, budget domain, split only when a step truly needs one, and keep the reason in the architecture record.

The question worth leaving with a buyer: which step of your multi-agent architecture was split out because it had to have its own authority or its own spending ceiling? If no step can be named, then what the architecture currently solves is the demo, not production.

References

Datadog, “State of AI Engineering 2026”, telemetry through March 2026 — source for 59% single-service and 18% three-or-more calls, framework adoption rising from 9% to 18%, 70% running three or more models, the February 5% error rate with 60% rate limits, the March 2% rate and roughly 8.4 million rate-limit errors, and the recommendation to propagate context and traces across service boundaries with tools included in service maps