Preventing runaway AI costs from a single customer

The overnight bill

An agent gets stuck retrying a tool call that keeps timing out. Nobody is watching at 2am, so it keeps going until a rate limit or a crash stops it. Or simpler: one customer kicks off a bulk job sized for ten records and it runs against ten thousand. Either way, you find out the same way: a token bill that’s an order of magnitude larger than yesterday’s, discovered after the money is already spent.

Both failures share one root cause. The system checked whether the request was well-formed. It never checked whether this tenant could afford it, because nothing in the request path asks that question before the model gets called.

The fix is an order of operations

A balance check that happens after the call has already been billed by the provider is not a safeguard. It’s a report. The check has to happen at admission, before execution starts, and it has to be able to say no. The specification calls this a gate, and it requires the mode to be declared rather than assumed:

Shadow mode is a legitimate rollout state, but it is not protection, and the specification is direct about why: a gate that does not stop spending has not refused anything. Only enforcingmode changes what happens to a request that shouldn’t run.

What a refusal actually looks like

A tenant with a $50.00 balance submits a turn whose class is priced at $62.00. At admission, before any model is called, the gate checks the reservation against the balance and refuses:

{ "turn_id": "turn_01JCMA",
  "outcome": "refused",
  "billable": false,
  "billable_reason": "refused_insufficient_funds",
  "cost_total": 0.00
}

No inference call happened. The refusal itself is still written as a record: a refused turn is an outcome, not a gap in the log. But nothing was spent producing it.

A hold is not the same as a check

A balance check alone has a race: two requests can both pass the check before either one debits the balance. The stronger version reserves funds at admission and resolves the reservation exactly once, either as a commitment when the turn emits, or a release when it doesn’t. A turn that fails after the reservation but before emission has to release the hold, not leave it stranded, and the usage record and the balance movement have to commit together, in one transaction. Otherwise a crash between the two leaves a debit with no matching turn, or a turn with no debit.

What this caps, and what it doesn’t

This doesn’t predict a runaway loop before it starts. It caps the damage at what the tenant has already funded: a stuck retry loop burns down to a zero balance, and every subsequent turn after it is refused in real time. The blast radius of a bad night becomes one customer’s prepaid balance, not your margin.

Gate mode and the reservation invariants are defined in turn-accounting §6.7, Gate mode and the Level 3 conformance tests T21–T23.