How to Reduce LLM API Costs

Measure where the tokens go

You can't cut what you can't see. Before you change models or prompts, break your spending down by feature, endpoint or agent, and split each into input tokens, cached input tokens and output tokens. Most providers and gateways report these per request; collect them alongside a feature or session label.

Two patterns show up again and again. A few features account for most of the spend, and in those features input tokens dominate, because the same instructions, history and documents are resent on every call. Start with the biggest line items and the parts of each prompt that never change.

Track one more number: cost per successful task, not just cost per request. A cheaper request that fails and has to be retried is not a saving.

Match the model to the task

The single biggest lever is not paying premium-model rates for work that doesn't need a premium model. Classification, extraction, short summaries, lookups and small code edits usually run well on efficient models. Multi-step reasoning, architecture questions and hard debugging justify a more capable one.

You can do this by hand: assign a model per feature and revisit the choice when new models ship. Or you can let an LLM router decide per request, which helps most when difficulty varies within a single feature, as it does in agents. Either way, keep a set of real test prompts and check that quality holds on the efficient model before you switch.

Send less context

Input tokens are billed on every call, so anything you resend unnecessarily is paid for again and again. Common sources of waste:

• Full conversation history when only the last few turns matter. Summarise or drop older turns.
• Whole documents or files when a relevant excerpt would do. Retrieve the passages you need instead.
• Verbose tool output, such as full test logs, large JSON payloads and directory listings. Filter it down to what the model must act on.
• Duplicated instructions spread across system prompts, templates and examples. Write them once.
• Unused tool definitions. Every tool schema you attach is part of the prompt. Attach the tools a request can actually use.

Coding agents are the extreme case, because they resend repository context and tool history at every step. How coding agents use tokens breaks this down.

Use provider prompt caching

Most major providers can cache a repeated prompt prefix and bill the cached part at a lower input rate. The rules differ by provider. Some apply caching automatically to repeated prefixes, while Anthropic's models need explicit cache_control markers in the request. Check your provider's documentation for minimum sizes, lifetimes and prices.

A few habits make caching work:

• Put stable content first. System instructions, tool definitions and reference material go at the start; the changing user turn goes last. Any change early in the prompt invalidates everything after it.
• Don't switch models casually. A cache belongs to one model. Anthropic's Claude Code documentation notes that switching models forces a fresh cache, so a "cheaper per token" switch can cost more once the cache rebuild is counted.
• Keep the hints intact. If a proxy or gateway strips caching markers, every turn bills as uncached input. Claude Code's gateway documentation calls this out explicitly.

Reuse responses for repeated requests

Prompt caching discounts the input; response caching skips the call entirely. If your application sends identical requests, such as the same FAQ question, the same classification of the same text or the same deterministic transformation, an exact-match response cache can answer from storage. Gateways such as LiteLLM and OpenRouter document response caches, and some offer semantic matching for near-duplicates.

Use it only where a repeated answer is correct. Anything time-sensitive, personalised or meant to vary should bypass the cache, and semantic caches need a strict similarity threshold so a different question doesn't get a stale answer.

Limit output and batch work that can wait

Cap output length

Output tokens are usually priced higher than input tokens, and long answers also take longer. Set a sensible maximum output length per feature, and ask for the shape you need:

• A diff or the changed function instead of a whole rewritten file.
• Structured output, such as JSON with named fields, instead of prose you then parse.
• A direct answer without restating the question or adding unrequested explanation.

For reasoning models, check whether your provider lets you set the reasoning effort or a thinking budget. Lower settings for routine tasks keep the hidden reasoning tokens in check.

Batch work that can wait

Not every request needs an answer in seconds. Overnight evaluations, bulk tagging, document processing and report generation can often go through a provider's batch or asynchronous endpoint, where one is offered. These endpoints trade speed for different pricing and limits; check your provider's terms for turnaround times and rates before you move a workload.

Even without a batch API, grouping small items into one request, such as classifying twenty short texts in one call instead of twenty, avoids resending the same instructions twenty times.

Set budgets and stop runaway loops

Set budgets and alerts

Budgets turn a surprise bill into an alert. Set limits at the level where spending is decided: per API key, per team, per feature or per environment. Gateways commonly support per-key credit limits and team budgets, and some agent tools have their own caps; Claude Code, for example, documents a maximum-budget flag for scripted runs.

Treat budgets as a safety net, not a hard stop. Most are enforced on a best-effort basis around request boundaries, so a long-running job can still overshoot slightly. Pair them with alerts that fire well before the limit.

Stop runaway agent loops

Agents add a failure mode that plain applications don't have: they can repeat themselves. An agent that keeps searching for the same file, re-running the same failing test or retrying the same edit makes a model call every time, each one carrying the full context.

Defend against this at several levels: cap iterations and wall-clock time per task, cap spend per run, and detect repeated actions. AI agent cost optimization covers why multi-step agents cost more than chat and how routing each step helps.

Cut cost without cutting quality

Every technique above can backfire if quality drops: retries and failed tasks quickly eat the savings. Keep the changes safe:

1. Build a small evaluation set from real requests, including hard ones.
2. Change one thing at a time, such as the model, the context or the output limit, and compare results.
3. Watch success rate and cost per successful task, not just tokens.
4. Roll out gradually and keep a way back to the previous setup.

Where Air Router helps

Air Router applies several of these levers for you at the routing layer. It routes each request or agent step to a suitable model, considers existing prompt caches before switching models, reduces repeated file content and completed tool history before the model runs, and helps stop unproductive agent loops. Usage shows in your AirLLM dashboard.

You connect it by changing the base URL, the API key and the model name in the tool you already use. If your main bill is a coding agent, start with how to reduce Claude Code costs; for plans and usage pricing, see AirLLM pricing.

Frequently asked questions

Related

Sources

• OpenRouter docs — Prompt caching
• OpenRouter docs — Response caching
• LiteLLM docs — Caching
• Claude Code docs — Prompt caching
• Claude Code docs — LLM gateway protocol
• Claude Code docs — Manage costs effectively