Most budgets for an AI feature are built from one number off a pricing page, multiplied by a guess. The result is usually wrong in both directions at once: the model call is cheaper than people fear, and the thing around it costs more than anyone put in the spreadsheet.
Here is the arithmetic, using published US rates as of September 2026, with the sources listed at the end.
The published rates, and the pattern inside them
Two things are worth noticing before any modelling. First, output costs about five times input on essentially every model from both major US providers. Second, a cache read costs roughly a tenth of a normal input token.
| Model | Input / 1M | Output / 1M | Ratio |
|---|---|---|---|
| Claude Haiku 4.5 | $1.00 | $5.00 | 5x |
| Claude Sonnet 5 | $2.00 | $10.00 | 5x |
| Claude Opus 5 | $5.00 | $25.00 | 5x |
| GPT-5.6 Luna | $0.20 | $1.20 | 6x |
| GPT-5.6 Terra | $2.00 | $12.00 | 6x |
| GPT-5.6 Sol | $4.00 | $20.00 | 5x |
Published list rates, September 2026. They change often, and they have mostly gone down. The ratios have not.
That ratio has a design consequence. A feature that reads a lot and writes a little is cheap. A feature that generates long documents is not, and asking for a shorter answer is a real cost control rather than a stylistic preference.
A worked example you can copy
Take an internal assistant over a company document library. It is the most common first build, and it makes the arithmetic concrete.
Multiply out. Input is 4.9 million tokens a day, output is 0.6 million. On Claude Sonnet 5 that is $9.80 of input and $6.00 of output, so $15.80 a day, about $474 a month.
Now hold the workload completely still and change only the model.
This is the single most useful chart in any AI budget conversation. The question is never “what does AI cost”. It is “which of these five is good enough for this job”, and that is answerable by testing rather than arguing.
The honest follow-up is that the cheapest model is not always the cheapest outcome. A weaker model that needs two attempts, or that produces answers people stop trusting, costs more than the difference. Pick the cheapest tier that passes your evaluation set, not the cheapest tier on the page. Deciding how to measure “passes” is its own problem, covered in testing software that answers differently every time.
What the model bill leaves out
The $474 is the part with a published price. Here is the rest of the monthly invoice for the same feature.
| Line | What drives it | Monthly |
|---|---|---|
| Model calls | Traffic and context size | $474 |
| Vector store | A plan floor, then storage and reads | $50 |
| Embeddings | Corpus size, at $0.02 per million tokens | <$1 |
| Tracing and evaluation | Per trace or per seat, varies widely | no list price |
| Human review | Someone reading a sample of answers | staff time |
| Visible total | The part with a published rate | ~$525 |
Embeddings surprise people by being negligible. A 50,000-document corpus averaging 800 tokens is 40 million tokens, which at $0.02 per million is eighty cents to embed the entire library. Re-embedding it every month still costs less than a sandwich.
The vector store behaves differently. Managed services start at a plan floor, commonly around $50 a month once you leave the free tier, then meter storage, reads and egress on top. Read cost scales with how much data a query has to scan, so a large single namespace is more expensive to query than several small ones. That floor is owed on a quiet month with no users, which makes it the first line to question if the feature is a pilot rather than a product.
The two levers that actually move it
Context size. Go from eight retrieved passages to sixteen and input rises from 4,900 to 8,900 tokens per request. On Sonnet 5 the monthly bill goes from $474 to about $714, a 51% increase, for a change nobody outside the team would notice. Retrieval quality is a cost decision as much as an accuracy one, and “retrieve fewer, better passages” is the highest-leverage tuning available. There is more on that trade-off in RAG, fine-tuning or prompting.
Model tier. The chart above. Route by task rather than picking one model for everything: classification and extraction on the cheap tier, drafting and reasoning on the expensive one. Most production systems that stay affordable do this.
Caching, and when it does nothing
Prompt caching lets a repeated prefix be charged at the read rate rather than the input rate, roughly a tenth. Writing to the cache costs a premium over normal input, commonly about 1.25 times, so one write pays for itself on the first reuse.
The catch is what fraction of your prompt actually repeats. In the example above the repeated part is an 800-token system prompt, and caching it saves around $43 a month out of $474, under ten per cent, because the bulk of the input is retrieved passages that differ on every question. Cache a 20,000-token policy document that every request carries and the same mechanism saves closer to $1,000 a month.
Caches expire. If your traffic is sparser than the cache lifetime, every request pays the write premium and few ever collect the read discount, so caching makes a low-volume feature slightly more expensive. Check your request spacing against the cache time to live before assuming it is free money.
The surprises that land later
Agent loops multiply everything
One user action that triggers a chain of tool calls is not one request. A five-step agent is five billable round trips, and each step usually carries the whole conversation so far as input. Costs grow faster than linearly with steps. Cap the loop and log the step count before you ship it, not after the invoice.
Residency and speed carry multipliers
Constraining inference to a specific region is priced as a premium, currently around 1.1 times list with Anthropic, and low-latency modes can run at double. Neither is unreasonable. Both are easy to agree to in a compliance conversation and then find on a bill nobody re-modelled. Model the multiplier at the point you accept the constraint.
Retries are real traffic
Failed parses, timeouts and validation rejections are charged. A feature that silently retries twice on malformed output is running at three times the volume you measured in testing. Count attempts, not successes, in whatever dashboard you use to forecast.
Anything asynchronous should be batched
Both major providers discount batch processing by 50%. Nightly classification, bulk summarisation and backfills have no reason to run at interactive rates. Ask which workloads genuinely need an answer in seconds. Usually fewer than the first design assumed.
How to budget this without guessing
- Estimate tokens, not dollars. Write down input tokens per request, output tokens per request and requests per day. Those three numbers, with published rates, give a cost you can defend and re-run when rates change.
- Price the top and bottom of the range. Cheapest viable tier and most expensive plausible tier. If both are affordable, stop optimising and go build. If only one is, you have found the real engineering constraint early.
- Add the floor. Vector store plan minimum plus tooling, owed in a month with zero usage.
- Put a number on the invisible lines. Even a rough allowance for evaluation, review and maintenance beats leaving them at zero, which is what makes year-one AI budgets wrong.
- Instrument from day one. Log tokens in, tokens out, model and attempt count per request. Retrofitting that after a surprise invoice is slow and you lose the history.
Rates fall. Between early 2025 and early 2026 published prices dropped sharply across the market, and the direction has been consistently downward since. Build the arithmetic so the rate is a variable you can change in one place, and this becomes a maintenance task rather than a re-estimate.
More on choosing where this effort is worth spending at all in where AI actually pays off in business software, and on keeping the data inside your own boundary in building an AI feature on your own data without leaking it.
We build and run these systems, including the unglamorous parts: token accounting, evaluation sets and the routing logic that keeps the cheap model doing the cheap work. Our AI and machine learning work covers it. If you want your own workload costed properly before anyone writes code, send us the numbers.
Common questions
How much does it cost to run an LLM feature in production?
At 1,000 requests a day with roughly 4,900 input and 600 output tokens each, published September 2026 US rates put the model calls between about $51 and $1,185 a month depending purely on model choice. Add a vector store floor of around $50, negligible embedding costs, and unpriced lines for evaluation tooling, human review and engineering time, which in year one usually exceed the API bill.
Why are output tokens more expensive than input tokens?
Generation is sequential and each output token requires a full forward pass, while input can be processed in parallel. The practical effect is consistent: across both major US providers, output is priced at five to six times input on every tier, so shorter answers are a genuine cost control.
Does prompt caching reduce LLM costs?
Only in proportion to how much of your prompt repeats. Caching an 800-token system prompt in a request that carries 4,900 input tokens saves under ten per cent. Caching a large fixed document that every request includes can save most of the input cost. If requests arrive less often than the cache expires, caching can cost slightly more than not caching.
What is the cheapest way to run a RAG system?
Retrieve fewer and better passages, route simple tasks to a cheaper model tier, batch anything that does not need an interactive answer for a 50% discount, and cap agent loops. Context size and model tier account for nearly all the variance. Embeddings and storage are rounding errors by comparison.
Sources
- Anthropic, Claude API pricing, retrieved September 2026: Haiku 4.5 $1 / $5, Sonnet 5 $2 / $10, Opus 5 $5 / $25 per million input / output tokens. Cache reads at roughly one tenth of input and cache writes at about 1.25 times input. 50% batch processing discount. US-only inference at 1.1 times standard pricing.
- OpenAI, API pricing documentation, retrieved September 2026: GPT-5.6 Luna $0.20 / $1.20, GPT-5.6 Terra $2.00 / $12.00, GPT-5.6 Sol $4.00 / $20.00 per million input / output tokens. Text-embedding-3-small at $0.02 and text-embedding-3-large at $0.13 per million tokens. 50% Batch API discount.
- Pinecone, serverless cost documentation, retrieved September 2026: plan minimums of $0 (Starter), $20 (Builder), $50 (Standard) and $500 (Enterprise) per month, with read units scaling by namespace size, write units by request size, and metered storage and egress.
- All monthly figures above are our own arithmetic on those published rates at the stated volumes, assuming a 30-day month. List prices change frequently and have trended downward. Re-run the calculation rather than quoting these numbers later.
Written September 2026.
