How to Estimate LLM Costs Before You Build: A Capacity-Planning Guide

The most common LLM cost mistake isn't picking the wrong model — it's not estimating the bill at all until after launch, when usage patterns are already locked into the product. This is a capacity-planning framework you can run in twenty minutes before you write a line of code, using nothing but a rough sense of your expected traffic.

The core formula

Every LLM cost estimate reduces to the same shape:

monthly cost = prompts/month × (input tokens × input rate + output tokens × output rate)

Where prompts/month = monthly active users × prompts per user per day × 30. Everything else is filling in these five numbers, which is exactly what the calculator automates once you have rough estimates — but understanding the shape of the math matters more than the tool, because it tells you which number to interrogate when the total looks wrong.

Step 1: Estimate traffic, not just users

"Monthly active users" is the wrong starting unit on its own — a user who logs in once a month and a user who sends 50 prompts a day produce wildly different bills. Break it down:

• MAU — how many distinct users will touch the feature monthly.
• Prompts per user per day — be honest about actual usage, not aspirational usage. A chatbot embedded in a support flow might see 2–5 prompts per active session; an agentic coding assistant used continuously by a developer might see 50–200.
• Multiply and ×30 for a monthly prompt volume. This number alone, before any pricing is applied, tells you whether you're building a $50/month side feature or a $50,000/month core product cost.

Step 2: Estimate tokens per prompt — separately for input and output

Because input and output are priced differently (often 3–8×, see Input vs Output Tokens), lump-summing "tokens per request" hides the number that actually drives cost. Estimate them separately:

• Input tokens — system prompt + retrieved context/documents + conversation history + the user's message. Use the token counter on a representative real prompt rather than guessing — teams routinely underestimate this by 2–3× once retrieved context and conversation history are included.
• Output tokens — your typical response length. A short chat reply might be 100–300 tokens; a generated report or code diff might be 1,000+.

Step 3: Apply a cache-hit assumption if relevant

If any meaningful fraction of your input is a repeated prefix (system prompt, tool schema, a reused document), estimate a realistic cache-hit rate and apply the discount — see Prompt Caching Explained for the mechanics. Getting this roughly right matters: a 0% vs. 50% cache-hit assumption can change your input-cost estimate by 2× or more.

Step 4: Add a buffer for retries and tool-use overhead

Real production traffic isn't as clean as the formula above. Add headroom for:

• Retries — failed generations, rate-limit backoffs, or validation failures that trigger a second call. 10–20% overhead is a reasonable starting assumption for most applications; agentic systems with tool-calling loops should assume more.
• Multi-step agent loops — if your feature makes several model calls per user-visible action (planning, tool call, verification), multiply your per-prompt estimate by the average number of calls per action, not per user message.

Three worked scenarios

Support chatbot. 5,000 MAU, 6 prompts/user/day, 1,200 input tokens (short conversation + a knowledge-base snippet), 300 output tokens, 40% cache hit rate on the repeated system prompt and KB snippet. → moderate, input-heavy volume where caching meaningfully reduces the bill. Model this on the calculator with the "RAG / search" preset as a starting point.

RAG document search. 3,000 MAU, 10 prompts/user/day, 6,000 input tokens (larger retrieved context), 400 output tokens, 50% cache hit rate. → clearly input-dominated; the input rate and cache-hit assumption matter far more here than model choice on the output side.

Autonomous coding agent. 1,000 MAU (developers), 15 prompts/user/day, but each "prompt" is really an agent loop averaging 4 model calls (plan → code → verify → fix), 2,500 input tokens and 1,500 output tokens per call, 25% cache hit rate. → output-heavy and call-volume-heavy; the model's output rate and your calls-per-task multiplier dominate the estimate far more than the headline per-token price.

Each of these has a matching preset on the calculator — start from the closest one and adjust the numbers to your actual expected traffic.

What to do with the number

Once you have a monthly estimate, stress-test it two ways: 10× the MAU (does this still work if the feature succeeds beyond plan?) and compare 2–3 models on the compare page at the same traffic assumptions, so a model choice is a business decision made with real numbers rather than a default. If the number is uncomfortable, revisit the cost-optimization playbook before you conclude the feature isn't viable — caching, batching, and tiered routing routinely cut a first-pass estimate by more than half.

The takeaway

A rough capacity-planning pass — five numbers, twenty minutes — before you build catches cost problems while they're still a spreadsheet exercise instead of a production incident. Run your own traffic assumptions through the calculator, and revisit the estimate once you have real usage data; first-pass guesses are often wrong in a specific, correctable direction (usually underestimating input tokens from retrieved context).