Run One Unit Before You Scale
Running an AI batch job? Measure the cost of one item before you run ten thousand. Money spent on an API call is as real as an outage, and far quieter: nothing pages you, the job just finishes and the invoice arrives. A batch of thousands multiplies whatever your estimate got wrong — so the cheapest place to catch a mistake is at unit number one.
The math is unforgiving because it's linear
Batch cost is almost perfectly linear: total ≈ per-unit cost × volume. There's no economy of scale hiding in the middle to save you. If one call is 2x pricier than you guessed, ten thousand calls are 2x pricier too. That linearity cuts both ways — it means a single measured unit predicts the whole run.
Anthropic's own pricing docs work a concrete example: classifying 10,000 support tickets at ~3,700 tokens each on Claude Haiku 4.5 ($1/M input, $5/M output) costs about $37.00. Swap the model to Claude Opus 4.8 ($5/M input, $25/M output) and the same 10,000 tickets cost roughly 5x more — same code, same prompts, one config line. That's the number you want to see before you press go, not after.
The three-step habit
- Estimate. Write down
per-unit cost × volumefrom the model's price card. Even a rough token count gets you to the right order of magnitude. - Run exactly one unit. Send a single real item and read the
usageblock in the response (input tokens, output tokens). Multiply by the price card to get the actual per-unit cost. - Reconcile, then scale. If the measured cost matches the estimate, scale. If it's 3x higher, you just saved yourself a 3x-over batch — find out why (a fat system prompt, an unbounded output, a retry loop) first.
A bug caught at step 2 produces a one-unit bill. The same bug caught after the full run produces a $500 one.
Rules of thumb
| Situation | What to check on the one unit |
|---|---|
| Long shared prompt | Cache it — a cache hit reads at ~10% of input price |
| Output length varies | Set max_tokens; unbounded output is unbounded cost |
| Retries on failure | Cap attempts — a retry storm multiplies the bill silently |
| Non-urgent job | Use the Batch API — 50% off input and output tokens |
| Any automated job | Bound the batch size so a bug stays cheap |
Power user: batch discounts and real caps
If the results don't need to be instant, both major providers offer an asynchronous Batch API at a flat 50% discount on input and output tokens. OpenAI's batches accept up to 50,000 requests (200 MB) and complete within 24 hours. Anthropic's Message Batches accept up to 100,000 requests (256 MB), with most finishing in under an hour. For evals, classification, and offline enrichment, that's half price for waiting a little.
One trap: platform "monthly budget" settings are increasingly alerts, not hard stops — they email you and keep serving traffic. Don't rely on them as a safety net. The real cap is the one you write into the job: bound the number of items per run, so the worst case is per-unit cost × your cap, not per-unit cost × infinity. Measure one, cap the rest.
Resources
- OpenAI Batch API guide — 50% discount, 50k requests, 24h window
- Anthropic Message Batches — 50% off, 100k/256 MB limits
- Anthropic pricing — per-token rates and the 10,000-ticket worked example
- OpenAI rate limits — throughput and usage-tier caps
Shipping AI at scale? Yeda AI designs, audits, and ships production LLM systems — with the cost math done up front.