How to Cut LLM API Costs: 8 Levers That Actually Work

LLM API bills grow quietly. A prompt picks up a few extra paragraphs, chat history piles up, a feature launches to more users — and the monthly total doubles without anyone making a single big decision. The good news is that most of the cost is controlled by a handful of levers, and many can be pulled without hurting quality.

This guide covers eight of them, in roughly the order we’d try them, and finishes with a worked example.

About the numbers: all prices and percentages below are hypothetical, chosen to illustrate the method. Discounts for caching and batching vary by provider and change over time; check your provider’s current pricing.

First: measure where the money goes

You can’t optimise what you don’t measure. Before changing anything, log for each request:

• Feature or endpoint name
• Model used
• Input tokens, output tokens (from the API’s usage field)
• Cached tokens, if your provider reports them
• Latency and whether the request succeeded

Then aggregate cost by feature. It’s common to discover that one feature or one oversized prompt accounts for a large share of total spend. Start there.

Remember the basic formula from our token cost guide: cost is input tokens × input price plus output tokens × output price. Every lever below reduces one of those four factors.

1. Shorten your prompts

System prompts tend to grow by accretion: every bug fix adds another rule. Review yours and remove:

• Duplicate or contradictory instructions
• Long explanations where one clear sentence works
• Few-shot examples that don’t measurably improve results (test with and without)
• Boilerplate the model follows anyway

Because the system prompt is sent on every request, trimming even a few hundred tokens adds up at volume. Run your eval set before and after to confirm quality holds.

2. Send less context

In retrieval systems, retrieving ten passages “to be safe” when three would do multiplies input cost. Tune the number and size of retrieved chunks against quality. In chat, trim or summarise older history instead of resending everything — see our context window guide for strategies.

3. Control output length

Output tokens are usually the most expensive tokens you buy. Ways to reduce them:

• Set a sensible maximum output tokens for each feature.
• Ask for the format you actually need: “Return only the JSON object” or “Answer in at most three sentences.”
• Avoid asking the model to restate the question or explain its reasoning when you only need the answer.
• For reasoning-capable models, check whether the provider lets you limit reasoning effort or budget for simpler tasks.

4. Use prompt caching

Many providers offer prompt caching: when consecutive requests share an identical prefix (a long system prompt, tool definitions, a reference document), the repeated part can be billed at a reduced rate and processed faster. Implementation details differ — some providers cache automatically, others require you to mark cache breakpoints, and caches expire after a period of inactivity.

To benefit:

• Put stable content (instructions, tools, documents) first, and variable content (the user’s message) last.
• Keep the stable prefix byte-for-byte identical across requests — even a timestamp at the top breaks the cache.
• Check the cached-token count in responses to confirm it’s working.

5. Batch non-urgent work

If results aren’t needed in real time — nightly classification, bulk summarisation, evaluation runs, data enrichment — several providers offer batch APIs that process requests asynchronously at a discount, typically with results returned within hours. Moving offline workloads from real-time to batch is often one of the easiest savings available.

6. Use a smaller model where it’s good enough

The biggest price differences are between model tiers. Smaller models can be a fraction of the price of flagship models, and for many tasks — classification, extraction, short rewrites, routing — they perform well. The only way to know is to test on your own examples.

7. Route requests by difficulty

You don’t have to pick one model for everything. A router sends easy requests to a small model and hard ones to a larger model. Routing can be:

• Rule-based: by feature, input length or user tier.
• Classifier-based: a cheap model or simple classifier labels difficulty first.
• Cascade: try the small model; escalate if its output fails validation or a confidence check.

Track the share of requests going to each tier and the quality on each path.

8. Cache responses and avoid waste

• Response caching: if many users ask the same question, store and reuse the answer at the application level.
• Fix retry storms: retries on timeouts or invalid output still consume tokens. Use structured output features and validation to reduce failures, and use backoff.
• Deduplicate: don’t re-process documents that haven’t changed.

Worked example (hypothetical)

Take the support assistant from our token guide: 5,000 requests per day on a hypothetical “Model L” at $2.00 per 1M input tokens and $8.00 per 1M output tokens, with 3,000 input and 400 output tokens per request. A hypothetical “Model S” costs $0.25 / $1.00.

Step	Change	Cost per request	Monthly (30 d)
Baseline	3,000 in / 400 out on Model L	$0.00920	$1,380.00
+ Trim prompt & history	2,200 in / 400 out	$0.00760	$1,140.00
+ Concise output	2,200 in / 300 out	$0.00680	$1,020.00
+ Route 60% to Model S	Blended L 40% / S 60%	$0.00323	$484.50

The arithmetic for the last row: Model S costs $0.00085 per request at 2,200 in / 300 out, so the blend is 0.4 × $0.0068 + 0.6 × $0.00085 = $0.00323.

In this illustration, the three changes together cut cost by roughly 65% — before applying any caching or batch discounts. Your results will differ; the point is that the levers compound, and you can model them on paper before touching production.

Don’t trade away quality blindly

Each lever is a hypothesis: “this change saves money without hurting results.” Test it:

1. Run your eval set before and after each change.
2. Roll out to a fraction of traffic first.
3. Watch user-facing signals: complaints, escalations, thumbs-down, retries.

Cost savings that quietly degrade answers usually cost more than they save.

Checklist

• Per-feature cost logging in place
• System prompt reviewed and trimmed
• Retrieval and history sized deliberately
• Max output tokens set per feature
• Stable content first to enable prompt caching
• Offline jobs moved to batch where supported
• Smaller models evaluated for simple tasks
• Retries and duplicate work minimised

MaximumAI will let you model these levers — prompt size, output length, routing split, caching — in the cost calculator, so you can see the effect before you change any code.