Español

Count Your Model Calls Per Turn

One question. Five model calls. You're paying for four you never see. When an agent gets a malformed or invalid answer, the wrapper code around your model can silently re-ask — again and again — until something validates. Those hidden loops burn money, add latency, and mask the real bug behind a bill.

Where the extra calls come from

Most modern agent stacks do a retry you never wrote. It's usually one of three:

• Structured-output re-asks. Validation libraries send the model a schema, and if the response fails to parse, they feed the error back and ask again. In Instructor, max_retries controls exactly how many extra round-trips this costs — each retry is a fresh model API call.
• Framework-level transport retries. LangChain retries model calls up to 6 times by default with exponential backoff before it gives up. A rate-limited or flaky call quietly becomes several.
• Agent tool loops. Every tool call an agent makes is another model call to decide the next step. A five-step plan is at least five calls, before any retries.

Stack two of these — a structured-output re-ask inside a framework that also retries — and one "turn" quietly multiplies.

Count the real number

Don't guess. Instrument the boundary where your code calls the model and log one line per call with a turn ID. Then count.

calls_this_turn = 0

def call_model(**kwargs):
    global calls_this_turn
    calls_this_turn += 1
    resp = client.chat.completions.create(**kwargs)
    log.info("model_call", turn=turn_id, n=calls_this_turn,
             tokens=resp.usage.total_tokens)
    return resp

Every provider returns a per-response usage object, so you also get tokens per call for free. If you're on an agent SDK, use its built-in accounting: the OpenAI Agents SDK exposes request_usage_entries, a per-request breakdown within a single turn — a run with entries [100K, 150K, 80K] made three model calls, not one.

Fix it: bound it, log it, surface it

A silent re-ask isn't resilience — it's a bug detector you've muted. Replace it with a retry that is:

1. Bounded. Set an explicit cap. In Instructor that's max_retries=2; in LangChain, drop the outer max_retries to a number you chose instead of the default 6. Never leave it unbounded.
2. Logged. Emit the failure reason on every retry, not just the final success. The interesting data is why attempt one failed.
3. Surfaced. When the cap is hit, raise — don't return a degraded answer as if it were fine. A failure you can see is cheaper than a failure you pay for silently.

Power user: put a budget on the turn

Set a hard call budget per turn (say, 3) and trip an alert when a turn exceeds it. Combine call count with token usage to catch the expensive case: a turn that's within its call budget but 10x the token cost is a runaway context, not a loop. Ship both numbers to your dashboards from day one — retrofitting cost visibility after the invoice arrives is how teams discover they were paying 5x all along.

Resources

• Instructor — Retrying — how max_retries turns validation failures into extra model calls
• LangChain — Structured output — the tool-calling round-trips behind with_structured_output
• LangChain — model_retry middleware — configurable retry-on-exception with backoff
• OpenAI Agents SDK — Usage — request_usage_entries counts real calls per turn