Español

Cost Per Task, Not Per Token

The pricing page lies to you. When a new model ships, the first thing most people check is the sticker: dollars per million input tokens, dollars per million output tokens. A cheaper number looks like a cheaper model. It usually isn't. The number that actually leaves your account at the end of the month is cost per completed task, and cheaper-per-token models routinely lose on it.

Tokens are the unit, not the cost

A token is a chunk of text a model reads or writes — roughly a word, sometimes a fragment of one (the word "programmatic" splits into "program" and "matic"). Models are billed per million tokens in and per million tokens out, and output tokens are the expensive ones: on a frontier model, output can run six times the price of input. That split matters, because the stage where a model writes a lot — actually generating code — is where the meter runs hottest.

Here's the trap. Sticker price compares tokens as if every token does the same amount of work. It doesn't. In one independent comparison, an open model was priced around $3 per million input tokens against a frontier model's roughly $5 — cheaper on paper. But the open model used about twice as many tokens to think through and solve the same task. Half the price times double the tokens lands you back at roughly the same total. The gap you thought you were getting evaporates to pennies.

The number that matters: cost per task

Cheaper-per-token models that burn extra tokens to reach the same answer have a name in this discussion: token hogs. The metric that sees through them is cost per completed task — sometimes called intelligence density, or how much useful work you get per token. Independent trackers like Artificial Analysis publish it directly: they run a fixed benchmark and report what it costs each model to finish, not what its tokens list for.

On that measurement the ranking reshuffles. Against a much pricier closed frontier model, a cheap open model is a genuine bargain per task. But against a token-efficient frontier model, the "half price" open option lands within about 10% — and in some configurations the efficient model finishes the task for less. Same answer, fewer tokens, lower bill.

Time is a cost too

Money per task isn't the whole ledger. Speed is a currency you're also spending. In one head-to-head front-end build, an open model was cheaper per token yet took roughly 90 minutes and over 20 million tokens to produce its result, while a frontier model finished the comparable job in under 20 minutes on a fraction of the tokens. If a task that takes 30 minutes on one model takes an hour on another, you paid the difference in your own time — a cost no pricing page prints.

How to actually compare models

1. Ignore the per-token sticker as a first-order signal. It tells you the rate, not the bill.
2. Look up cost per task on an independent tracker (Artificial Analysis runs a standard suite) before you commit a model to real work.
3. Weigh wall-clock time. A model that's cheaper per task but twice as slow may still be the wrong pick for interactive or high-volume work.
4. Measure on your own workload. Benchmarks are a starting point; your prompts, your codebase, and your caching setup shift the numbers. Run a representative task through each candidate and read the total.

Takeaway

Never rank models by the pricing page. A token isn't a commodity — one model's token can carry far more finished work than another's, so a lower per-token rate can quietly cost you more per task and more in time. Compare the number you actually pay: cost per completed task, plus the clock.

Resources

• Plan Expensive, Build Cheap, Review Sharp — Yeda AI Tips #205
• Caching Is Where Open Models Get Cheap — Yeda AI Tips #207
• Artificial Analysis — cost-per-task benchmarks