AI agent timeout: failures are bimodal, not slow

How long does a failed tool call take? We went through 2.6 days of our own agent fleet logs, and the 423 failures answered in two piles: 0.4 seconds, or 180 seconds. Almost nothing in between.

What the 423 failures are

The sample is 423 tool calls that returned an error in OOMeta’s own fleet log, with durations read from each line’s own timing field, over a 2.6-day window from 21 to 23 September 2026. One caveat matters: this file records failures only, so successful call durations are absent. What follows describes the shape of the failures, not of all traffic.

Two modes, not a long tail

Laid out as percentiles: p25 is 0.35 seconds, the median is 1.63 seconds, p75 jumps to 37.96 seconds, p90 is 180.55 seconds, p95 is 182.11 seconds, and the longest single call logged 1,876.89 seconds.

As shares it is clearer: 53 percent of failures returned within 2 seconds, 16 percent sat at 170 seconds or more, and the middle accounted for 31 percent. That wide gap between the two clusters is the tell that these are not the same failure.

The fast mode is a command that was wrong: a missing path, a bad argument, a command blocked by a safety gate. The slow mode is a call that never returned: waiting on the network, on a lock, on input to an interactive command. Eighty-eight failures are written up as command timeouts, and 48 carry exit code 124, the system-level timeout verdict.

The two modes react to retries in opposite directions. A fast failure often clears on retry because the state was simply not ready. A cap-hit retry is wasted motion, because the previous attempt never came back and the next one will most likely wait in the same place.

Why the average is the worst input here

Set one global timeout from the average duration and both modes get misjudged. The fast cluster never needed that much time, and the slow cluster will not return no matter how long it waits. The average sits in the empty gap, which is precisely the least informative place to stand.

Our judgment

1. Use the cap as a classifier. Count cap-hit calls separately, and treat them as interrupt-and-redesign rather than retry-and-hope.
2. Budget per tool class, never one global number. Terminal commands, file reads and external APIs differ in time scale by two orders of magnitude. One number lets the slow ones through and makes the fast ones wait.
3. Write never returned into the runbook as its own failure class. Its response is the opposite of returned an error, and mixing them in one handler drags both down.

One question for buyers

How many timeout values does your agent have today? If the answer is one, two failure classes with opposite remedies are already sharing a single handler. Do not start by tuning the number. Split the two clusters into separate counters first, and you will know immediately whether the fix belongs in the retry policy or in the tool itself.

Caliber and sources: every figure comes from OOMeta’s own agent fleet runtime log, first-hand, with no third-party vendor data. To reproduce: the log file at ~/.hermes/logs/errors.log, retained window 2026-09-21 07:18 to 2026-09-23 20:01 CST (1.4 MB, 16,787 lines); the sample is the 423 lines matching Tool returned error, with the duration read from that line’s timing field; timeout counts come from the Command timed out string and from exit code 124; collected 2026-09-23 20:10 CST. Log rotation changes the sample size, so a fresh run will match this caliber rather than these exact numbers. External caliber references: the exit code 124 timeout semantics are documented in the GNU coreutils timeout manual at gnu.org/software/coreutils/manual/html_node/timeout-invocation.html (curl HTTP 200), and HTTP 429 is defined in rfc-editor.org/rfc/rfc6585 section 4 (curl HTTP 200). Both are terminology references; no figure here depends on third-party data.