What a turn costs: measuring AI agent cost end-to-end

The number that’s quietly wrong

A team reports $0.03 per conversation internally and prices around it. Margin doesn’t match six weeks later. Nobody lied about the number. It just never included the paid enrichment API the agent calls on every third request, or the guardrail evaluation that runs before every response, or the retries a flaky provider forced last Tuesday. The figure was real. It measured a component, not the turn.

Everything inside the boundary, not just the model call

End-to-end means every cost-bearing operation between admission and emission, not only the inference call in the middle of it. The specification names nine components an implementation must populate wherever it has knowledge of them: input tokens, cached input, cache writes, output tokens, reasoning tokens, guard evaluations, retrieval legs, tool execution, and infrastructure. One of those gets left out more than the rest:

tool_execution is required, and it is the component most often omitted. An agent that sends messages, calls a paid enrichment API, or triggers a payment incurs real cost outside inference. On many production workloads it exceeds the model cost.

A figure computed from token counts alone isn’t a rough approximation of the real number when tool cost is the larger half. It’s a measurement of the wrong thing, reported with the confidence of the right one.

A number needs to say how sure it is

Not every component is directly billed by a provider. Reasoning-token share is sometimes inferred rather than reported; infrastructure overhead is often modeled, not metered. That’s legitimate, provided it’s labeled. Every component carries a provenance:

That last row is the one teams get wrong by default. Recording an unknown cost as zero doesn’t just under-report one turn. It silently corrupts every total, average, and margin figure built on top of it, because a zero looks exactly like a component that genuinely cost nothing.

The denominator problem

A subtler failure sits beneath the cost figure entirely: which turns get counted at all. A system that only writes a record on the success path produces a distribution with every percentile computed over survivors. Refused, failed, and interrupted turns vanish from the denominator instead of pulling the average toward zero where they belong. The fix isn’t a smarter query, it’s an independent count of terminal outcomes, incremented at every exit path in the pipeline rather than read from the same store being measured, then compared against records actually written. The ratio is record coverage, and it’s the only measurement that catches a record that was never made.

Report the distribution, not the average

A mean cost per turn over a right-skewed distribution (the normal shape for agent cost, where a handful of long conversations cost far more than the median one) systematically understates what your most engaged customers actually cost you. Percentiles fix that, but only past a real sample size: a p95 needs at least ten observations above it to mean anything, which in practice means roughly 200 samples for a p95 and 1,000 for a p99. Below that, the number is an artifact of the two or three most expensive turns in the batch, and it will move by an order of magnitude with the next one.

The full component list, provenance rules, and the sample-size requirement for a published figure are in turn-accounting §5.2 and §10.1, Statistical adequacy.