Input vs Output Tokens: Why Output Pricing Quietly Drives Most of Your Bill
Ask most developers what an LLM API costs and they'll quote a single number. But every major provider charges two separate rates — one for the tokens you send (input) and a meaningfully higher one for the tokens the model generates (output) — and the gap between them is usually the single biggest lever on a bill that nobody looks at first.
Why generation costs more than reading
Processing input is largely parallel: the model can attend across the entire prompt in one pass. Generating output is sequential — each new token depends on every token before it, computed one at a time. That structural difference in compute cost is why output tokens are priced higher across essentially every provider, commonly 3–8× the input rate depending on the model.
This isn't a rounding difference. It means a request that generates a long response can cost more than one that reads a much longer document and returns a short answer, even though the second request "used more tokens" in a naive total-token count.
The number that actually predicts your bill
Total token count is a poor predictor of cost. Your input:output ratio is the number that matters, because it determines how much of your bill lands on the expensive side of the rate table.
| Workload shape | Typical ratio | What drives cost |
|---|---|---|
| RAG / document Q&A | 10:1 to 20:1 input-heavy | Input rate dominates |
| Chat / general assistant | roughly balanced | Both rates matter |
| Drafting / long-form generation | 1:3 to 1:10 output-heavy | Output rate dominates |
| Code generation / agents | often output-heavy | Output rate dominates, and multi-step loops compound it |
Two applications with identical total token volume can have wildly different bills if one is input-heavy and the other output-heavy — because they're paying different blended rates for the same token count.
A concrete illustration
Take a model priced illustratively at $1 per 1M input tokens and $6 per 1M output tokens — a 6× ratio, not unusual for a capable model. Compare two requests that each move 1,000 tokens total:
- 900 in / 100 out: (0.0009 × $1) + (0.0001 × $6) = $0.0015
- 100 in / 900 out: (0.0001 × $1) + (0.0009 × $6) = $0.0055
Same total token count, 3.7× difference in cost, purely from which side of the ratio the tokens landed on. At production volume this gap is the difference between a bill you budgeted for and one that surprises you.
What this means practically
- Cap output length deliberately. A max-tokens limit that's larger than necessary is one of the cheapest fixes available — most applications don't need the model to be able to generate an unbounded response. See the cost-optimization playbook for more levers.
- Choose models by ratio-weighted price, not sticker price. A model with a low input rate but a steep output rate can lose badly on an output-heavy workload even if it "looks cheap" — see Which Frontier Model Is Actually Cheapest for Your Workload? for the full framework.
- Prompt caching only discounts input, never output (see Prompt Caching Explained) — so for an output-heavy application, caching helps less than it would for an input-heavy one, and other levers matter more.
- Measure your real ratio before comparing models. Paste representative prompts into the token counter to see your actual input token count, and track your typical response length separately — guessing the ratio is the most common way teams misjudge which model is actually cheapest for them.
The takeaway
A pricing table with two numbers per model looks simple, but the interaction between those two numbers and your specific traffic shape is where the real cost lives. Know your input:output ratio, weight your model comparison by it, and treat output length as a cost lever you control — not a fixed cost of doing business. Run your own numbers on the calculator, which prices the input and output sides separately rather than collapsing them into one misleading average.