Learn · Developer

Choosing Models and Controlling Costs

The most expensive model is rarely the wrong choice for quality. It is frequently the wrong choice for the job.

Model names change every few months; the structure of the choice has not changed in years. Every major provider ships a ladder: a frontier model that is the smartest and priciest, a mid-tier workhorse, and a small, fast, cheap tier — with the reliable pattern that today's small model performs near yesterday's flagship. Internalize the ladder and specific releases become details you look up on [the current directory](/models/) rather than news you must keep up with. This guide is about matching rungs to jobs, and then about the levers that keep the bill proportionate to the value.

The four axes that decide it

Capability. Frontier models earn their price on genuinely hard work: multi-step reasoning, subtle writing, complex code, [agent loops](/build/building-agents/) where one wrong decision derails the run. Classification, extraction, formatting, routing, summarizing routine text — the volume work of most applications — sits comfortably in the smaller tiers.
Latency. Anything conversational or interactive lives or dies on response feel, and small models are dramatically snappier. A user waiting on an autocomplete does not care that the answer could have been five percent more brilliant.
Context. If the job means digesting very long documents in one pass, the context window is a hard constraint that shortlists your options before quality enters the debate.
Price, times volume. A call that runs once an hour can afford any model on the menu. A call that runs on every keystroke of every user cannot afford much of anything — scale, not sticker price, is what makes cost an engineering topic.

The method: measure down

Folk wisdom says start cheap and upgrade when quality disappoints. Production experience says the reverse is faster: prototype on a top-tier model so capability is never the confounder while you fix prompts and retrieval — then, once your [eval suite](/build/evals-and-testing/) exists, try the cheaper tiers against it. If the mid-tier clears your bar, you just cut costs severalfold with evidence instead of vibes. Rerun the exercise when providers ship new models, because the answer changes in your favor over time. Without evals this exercise is guessing; with them it is an afternoon.

Heterogeneous designs are the norm at maturity, not an exotic optimization. A pipeline might route with a small model, extract with a mid-tier one, and reserve the frontier tier for the one step that is actually hard. Cascades formalize it: let the cheap model attempt everything and escalate to the expensive one only on low confidence or failed validation — most inputs are easy, and paying frontier prices for easy inputs is the single most common form of AI overspend.

The levers on the bill

Trim what you send. [Tokens are the meter](/build/your-first-api-call/), and inputs dominate most bills. Resending an entire conversation or document when a summary or excerpt would do is the first thing to fix — attention to context length compounds across every call.
Prompt caching. Providers discount the repeated prefix of your prompts — the system prompt, the stuffed reference document — often dramatically. Structure prompts so the stable part comes first and the variable part last, and [context-stuffing designs](/build/building-with-rag/) get markedly cheaper.
Batch the non-urgent. Work that tolerates delay — nightly enrichment, backfills, report generation — frequently qualifies for batch pricing at a steep discount. If nobody is waiting on the answer, stop paying interactive prices for it.
Cap the loops. Agent iterations and retry logic are where costs go nonlinear; budgets and stop conditions are cost controls as much as safety ones.
Alert on the meter. Spending dashboards and alerts exist in every provider console. The failure mode they prevent — a runaway loop or a viral traffic day discovered on the invoice — is entirely generic and entirely avoidable.

Know your unit economics

The question that reframes every optimization: what does one unit of your product's work cost in tokens, and what is that unit worth? Cost per document processed, per conversation, per generated report. Sometimes the answer reveals a real problem — a feature whose token cost exceeds its price. At least as often it reveals the opposite: an heroic optimization effort aimed at a cost that was already trivial next to the value delivered. Model prices have also fallen relentlessly for equivalent capability, year over year — which argues for building the thing that is almost affordable now, and against contorting your architecture around a price that will not survive the next release cycle.

That closes the loop this track opened: [a call](/build/your-first-api-call/), [a contract](/build/prompting-for-programs/), [your data](/build/building-with-rag/), [tools and autonomy](/build/building-agents/), [proof it works](/build/evals-and-testing/), and now a bill that makes sense. What remains is the part no guide supplies — the thing only you know is worth building.

Common questions

How do I choose which AI model to use?

Score the job on four axes: how hard the task genuinely is, how much latency users will tolerate, how much context it requires, and price multiplied by call volume. Frontier models earn their cost on hard reasoning and agent work; classification, extraction, and routine summarization usually belong on smaller tiers.

Should I start with a cheap or expensive model?

Prototype on a top-tier model so capability never confounds your debugging, then use your eval suite to test cheaper tiers against your actual cases. If a cheaper model clears your bar, downgrade with evidence. Rerun the comparison as new models ship — the answer improves in your favor over time.

What are the biggest levers for reducing LLM costs?

Send less: trim inputs and conversation history, structure prompts so stable content leads and gets prompt-cached, batch non-urgent work at discounted rates, cascade easy inputs to small models with escalation on failure, and cap agent loops and retries. Then verify with spend alerts rather than the invoice.

What is a model cascade?

A design where a cheap model attempts every input and only low-confidence or validation-failing cases escalate to an expensive model. Because most real inputs are easy, cascades cut costs substantially while holding quality — the expensive tier is reserved for the minority of cases that need it.