Español

Put a Hard Per-Job Cap on Every Automated AI Call

Every automated AI call needs a hard cap. A bug should produce a bounded bill, not an unbounded one. When a human is in the loop, a runaway prompt is annoying — you notice, you stop it. When the call runs on a cron, in a queue worker, or inside an agent loop, nobody is watching each run, and the meter climbs quietly all night.

Why recurring spend is the dangerous kind

Interactive spend is self-limiting: a person is sitting there. Automated spend is not. The three classic failure modes each multiply your bill without anyone noticing:

• A loop that reruns. A retry with no backoff, or an agent that keeps calling itself, turns one job into thousands.
• A payload that balloons. Yesterday the input was 2 KB; today someone pasted a 4 MB log, and every downstream call inflates with it.
• An output that never stops. Without a token limit, a confused model can generate to the model's full context ceiling on every single call.

None of these throw an error. They just cost money until the invoice arrives.

Cap the three axes, then alarm the account

Put a hard cap on every axis a runaway job can grow along, at the call site — not in a config nobody reads.

On Anthropic's Messages API, max_tokens is a required parameter — you cannot make a call without declaring the ceiling. On OpenAI's Chat Completions, max_tokens was deprecated in September 2024 in favor of max_completion_tokens, which is the only form the o-series reasoning models accept. Set it deliberately to what the task actually needs, not to the maximum.

Capping tokens bounds one call. Capping items bounds the fan-out — a job that should touch 100 rows should refuse to touch 100,000. Capping time bounds a hung call. And a budget alarm is the backstop for everything you forgot: it watches the money directly.

The account-level backstop

Even a perfectly capped job can surprise you when ten of them run at once. Put a spend alarm on any account the automation can bill against. AWS Budgets, for example, lets you set a cost budget with a fixed monthly target and get notified on both actual (already accrued) and forecasted (projected) spend — a common pattern is an alert at 80% of the budget. Notifications go to email or an Amazon SNS topic. Budgets update up to three times a day, so treat the alarm as a floor, not a real-time kill switch.

Power moves

• Fail closed, not open. When an item count or elapsed time exceeds the cap, raise and stop the job — don't log a warning and keep going. A fence you can walk through isn't a fence.
• Add a budget action, not just a notification. AWS Budgets can attach an action at a threshold — for instance, applying a restrictive IAM policy that blocks new spend — so the cap enforces itself instead of waiting for a human to read the email.
• Log the cap, not just the usage. Record max_tokens, the item ceiling, and the timeout you sent on every call. When something looks wrong, you want to see the fence that held (or the one that was set too high).
• Test the runaway. Feed the job a deliberately huge payload in staging and confirm it hits the cap and stops. An untested fence is a hope.

Resources

• Anthropic Messages API — max_tokens is required
• OpenAI — controlling response length (max_completion_tokens)
• AWS Budgets — cost budgets and alerts
• AWS Budgets — configuring budget actions