LLM cost optimization: a measured playbook for cutting API spend

TL;DR

• Every LLM bill is requests x (input tokens x input price + output tokens x output price). Each technique below shrinks one factor: fewer requests, fewer tokens, or a lower price per token.
• Measure cost per feature before changing anything. Without attribution, teams routinely trim a feature that was a rounding error and miss the one burning the budget.
• The largest wins usually come from right-sizing models and routing by complexity, then from provider prompt caching with byte-stable prefixes, then from Batch APIs, which OpenAI, Anthropic and Google all price at 50 percent of standard rates.
• Output tokens cost five times input on most current flagships, so max_tokens, reasoning budgets and agent loop caps matter more than they look.
• The open-source llm-cost-reduction agent skill packages this playbook, including a seven-part grep audit, for Claude Code, Codex and Cursor.

LLM cost optimization is the practice of lowering what an application spends on model API calls while holding output quality steady, and it works in a fixed order: measure cost per feature, fix the largest line item, then prove quality did not drop. Smaller models cut the price per token, caching and batching cut the price or the request count, and trimming cuts the tokens. What follows works through each lever with provider pricing checked on October 28, 2026, and shows how an open-source agent skill turns the checklist into something a coding agent can run against a real repository. No production savings were measured for this piece; the numbers come from provider documentation.

The LLM cost equation

LLM cost optimization reduces one of three factors: how many requests are made, how many tokens each carries, or the price per token. Every technique in this playbook maps to one of them, and confusing them leads to wasted effort, such as caching responses for traffic that never repeats.

The mapping is worth keeping in view, because it also says what each fix cannot do:

Output is the expensive side. Anthropic’s model pricing table lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens, a five-to-one ratio, and OpenAI’s pricing page lists gpt-6.1-sol at the same $2 and $10. The Frontier Wire’s LLM API pricing comparison covers the full price sheets; this piece is about what to do with them.

Cost per feature: measure first

Cost attribution means every model call records which feature, team and model it served, plus its token counts, so spend can be ranked by owner. It comes first because the top one or two features usually account for most of the bill, and only data shows which ones they are.

The minimum record per call is feature, model served, input, output and cached tokens, and latency. The llm-cost-reduction skill’s first fix is a client wrapper, in Python and TypeScript, that logs those fields as JSON on every call. Its stated rule: “Without this you cannot prove any later fix worked.”

Three habits keep the numbers honest:

• Count everything that costs money. Embeddings, classifier calls, LLM-as-judge evals, retries and background jobs are billed like chat. The skill lists “counting only chat calls” among its common mistakes.
• Price at capture time, with the rates actually paid. Recomputing later loses which price applied, and list prices drift from an invoice with committed-use discounts.

Collect a day of traffic, then rank features by total cost and by cost per 1,000 requests. The sibling explainer on LLM observability covers the wider per-call record, including traces and latency, that cost attribution sits inside.

Model right-sizing

Model right-sizing means assigning each call site the smallest model that passes its quality bar. Classification, extraction and short rewrites rarely need a frontier model, and the price gap between tiers makes this one of the largest savings available.

Anthropic lists Claude Haiku 5.5 at $0.10 input and $0.50 output per million tokens for prompts up to 100,000 tokens, against $4 and $20 for Claude Opus 5.5: a forty-fold difference on both sides. OpenAI’s gpt-6-luna sits at $0.10 and $0.50 against $10 and $50 for gpt-6-astra.

The llm-cost-reduction skill classifies call sites into three shapes:

The safe procedure is an eval, not intuition. Build 50 to 200 real inputs per feature with expected outputs or a rubric, run the current model and the cheaper candidate, compare scores, and switch only the call sites that pass. The skill says its model IDs “are examples only,” and lists shipping a smaller model without an eval set among its common mistakes.

Complexity routing

Complexity routing classifies each request at run time and sends simple ones to a cheap model and hard ones to a frontier model. It fits endpoints such as chat boxes and coding assistants that receive both kinds of request, where no single static model choice is right.

The skill’s routing reference starts with a traffic study: sample 200 to 500 real requests, label each SIMPLE, MEDIUM or COMPLEX, test the cheaper models on their buckets, and estimate savings as share_simple x (price_frontier - price_small) + share_medium x (price_frontier - price_mid), minus classifier cost. If 70 percent of traffic is simple and the small model holds quality, routing pays; if almost everything is complex, trimming and caching are the better targets.

Two implementations are described. The app-side version is a classifier call to the small model with max_tokens=3 and a 1.5-second timeout that returns COMPLEX on any error, so a broken router fails toward quality. The gateway version uses Bifrost’s Complexity Router, which publishes a complexity_tier variable to routing rules without application changes. Its documentation states that classification “runs only when a routing rule references complexity_tier” and that if no tier is produced, the request “continues through normal routing.” Its default decision-model classifier typically adds 600 to 800 ms; the semantic classifier is usually faster.

Two pitfalls stand out. Prompt length does not predict difficulty. And a classifier that silently skips requests sends them to the model the app asked for, often the frontier model, removing all savings; the skill says to watch the skipped share in metrics. The Frontier Wire’s survey of LLM routers covers routing infrastructure more broadly.

Prompt and RAG context trimming

Prompt trimming cuts the input tokens paid on every call: system prompts, tool schemas, retrieved chunks, conversation history and tool results. Input is billed again on every turn, so a bloated prefix multiplies across a whole session. The skill’s checklist:

• System prompt: remove duplicated instructions, stale few-shot examples and boilerplate, and measure before and after with a tokenizer or the provider’s counting endpoint. Anthropic’s token counting endpoint accepts the same inputs as a message request, including tools, images and PDFs.
• RAG: retrieve fewer, better chunks. Lower top_k, add a reranker, deduplicate overlapping chunks, and strip markup before indexing.
• History: keep the last few turns verbatim and replace older ones with a running summary written by a small model.
• Tools: send only the tools relevant to the current step. Twenty schemas can cost thousands of tokens per call.
• Tool results: truncate or summarize logs, HTML and JSON dumps before appending them.

Trimming interacts with caching: a long cached prompt can cost less than a short one rebuilt every request, and rewriting history every turn breaks the cached prefix, which is why the skill suggests summarizing “in larger, infrequent steps.”

Provider prompt caching

Provider prompt caching bills a repeated prompt prefix at a fraction of the normal input price, because the provider reuses work it already did. It only hits when the prefix is byte-identical, so the engineering work is keeping static content first and stable. Terms checked on October 28, 2026:

Anthropic’s pricing page states that caching “pays off after one cache read for the 5-minute duration” and after two for the 1-hour duration, Its cache limitations section adds that shorter prompts are processed uncached with no error. OpenAI’s model comparison shows write charges only on GPT-5.6 and later.

The arithmetic can be large. Take an illustrative assistant on Claude Sonnet 5.5 with a 4,000-token static system prompt and tool block, called one million times a month. Uncached, that prefix costs $8,000 at $2 per million tokens. Read from cache at $0.10 per million, it costs $400, plus write charges each time the cache expires.

Getting hits means putting tools, system prompt and static material before history and the user message; keeping timestamps, request IDs and user names out of the system prompt; sorting JSON keys; and keeping a conversation on one provider, model and key. For a zero hit rate, the skill’s diagnosis is to diff the first 2,000 tokens of two consecutive requests, then check the minimum length, the provider and key, and the gap against the cache lifetime. Load balancing scatters a conversation across cold caches; Bifrost’s session affinity pins a session to the provider and key that served it, and its auto prompt caching adds a breakpoint for clients that send none. The Frontier Wire’s prompt caching guide goes deeper on every provider.

Exact and semantic response caching

Response caching skips the model call entirely when the same question arrives again. An exact cache matches a hash of the model, messages and parameters; a semantic cache embeds the request and serves a stored answer when similarity clears a threshold, catching paraphrases at the risk of wrong matches.

Exact caching is the safe first layer. The skill’s version hashes the request parameters with sorted keys, stores the response in Redis with a TTL, and skips responses containing tool calls, because replaying one can run a tool with someone else’s arguments.

Semantic caching raises the hit rate on FAQ and support traffic, and the stakes. The skill says to start the threshold at 0.8 to 0.9, sample semantic hits, and “raise the threshold the moment you see a hit that answers a different question.” Any response that depends on private data needs the user in the cache key or no cache at all.

Latency is a real cost. Bifrost’s semantic caching documentation notes that a semantic miss “pays the embedding call on top of the full LLM call, making it slower than running without the cache,” and that caching engages only on requests that carry an x-bf-cache-key header or when a default key is configured. The tradeoffs between the two cache types get a full treatment in prompt caching vs semantic caching.

Batch APIs and embedding reuse

Batch APIs process requests asynchronously at half the standard price, which makes them the easiest saving for work that can wait: evals, backfills, nightly enrichment, bulk classification and embeddings. Embedding reuse removes the other common offline waste, re-embedding text that has not changed.

The three major providers publish the same discount with different limits:

OpenAI’s Batch API guide promises “50% lower costs” and a separate pool of higher rate limits, with per-batch limits listed under rate limits. Anthropic’s batch documentation reports “most batches finishing in less than 1 hour” and keeps results for 29 days. Google’s Batch API page prices jobs “at 50% of the standard cost.” OpenAI also lists a Flex tier at Batch prices for synchronous requests that tolerate slower responses, and Anthropic’s cache multipliers stack on the batch discount. Batch is for work that can wait hours; the skill lists moving interactive traffic to it as a mistake. For gateway users, Bifrost exposes Files and Batch APIs across providers.

For embeddings, the skill keys stored vectors by sha256(model + normalized text), embeds only the misses in batched calls, and chunks deterministically so unchanged documents hash the same.

Output limits and agent loop caps

Output caps bound the most expensive tokens on the bill, and loop caps bound the most expensive failure mode: an agent that keeps calling tools. Both are small changes that teams skip until an incident, so the skill’s grep audit checks for them directly.

The output rules are simple. Set max_tokens (or max_completion_tokens, max_output_tokens, maxTokens) sized to the task: “A classifier needs single digits; a summary needs a few hundred.” Ask for the format needed, such as three bullets or a JSON schema, instead of prose that is parsed afterwards. For reasoning models, set the reasoning effort or thinking budget explicitly, since “the default can dominate the bill.”

Agents need caps on three axes at once: iterations, total tokens and wall time. The skill’s loop stops at 12 steps or 200,000 tokens, whichever comes first, and returns a partial result flagged for review:

MAX_STEPS, MAX_TOKENS_PER_RUN = 12, 200_000
used = 0
for step in range(MAX_STEPS):
    resp = llm_call("agent", model=MODEL, messages=msgs, tools=tools, max_tokens=1024)
    used += resp.usage.total_tokens
    if used > MAX_TOKENS_PER_RUN:
        break  # return partial result + flag for review
    if not resp.choices[0].message.tool_calls:
        break
    msgs += run_tools(resp)

Two smaller rules prevent quiet multiplication: cap retries at two or three with exponential backoff and jitter, never retrying 400-class errors, and stop an agent that makes the same tool call with the same arguments twice in a row. The Frontier Wire’s guide to rate limiting covers retry and 429 handling in more depth.

Budgets and cost attribution

Budgets turn attribution into enforcement: each team, feature or customer gets a spending limit that fails safely before the monthly invoice arrives. Attribution supplies the labels; the budget supplies the hard stop.

The skill’s attribution reference starts with label design: feature, team and environment are bounded and belong on metrics, while user, request and conversation IDs belong in logs and traces only. Without a gateway, it extends the usage wrapper with a price table, emits a llm_cost_usd_total{feature,team,model} counter, and enforces budgets with a Redis counter keyed by team and month. Its own caveat: every service must implement this identically, and price tables drift.

The gateway version centralizes all three. In Bifrost, each team or feature gets a virtual key with a budget such as {"max_limit": 100.00, "reset_duration": "1M"}, a rate limit and an allowed-model list. The budget and limits documentation describes budgets at virtual key, team and customer level. Exhausted budgets return HTTP 402 and rate limits return 429; the skill’s sample code catches the 402 and degrades to a fallback answer rather than retrying. A routing rule on budget_used > 80 can move traffic to a cheaper model before the limit hits. Custom pricing overrides catalog prices with negotiated rates, so reported cost and enforced budgets match the invoice.

For alerting, the skill suggests hourly spend by team or key, and rising 402 and 429 counts. The Frontier Wire’s piece on LLM cost control with an AI gateway compares provider, app-side and gateway approaches to budgets in detail.

The llm-cost-reduction skill

llm-cost-reduction is an MIT-licensed agent skill from the build-with-ai-hq GitHub organization that teaches a coding agent this playbook. It follows the Agent Skills format: a SKILL.md file whose name and description load at startup, with detail in reference files the agent reads only when a task needs them.

What it contains. The repository holds SKILL.md, a README, a license and three references: budgets-and-attribution.md, caching.md and complexity-routing.md. SKILL.md carries the cost equation, nine fixes in order of typical impact with Python and TypeScript code, a Bifrost gateway section, a verification list and common mistakes. It opens with a seven-check grep audit: every LLM call site and model ID, output-limit parameters, uncapped loops, retry wrappers, whether usage is recorded, timestamps or UUIDs near the system prompt, and embedding calls that may rerun on every ingest. Two of the checks:

g() { grep -rnE "$@" --exclude-dir={node_modules,.git,dist,build,.venv,venv,__pycache__} . ; }
g -o "\"(gpt-[0-9a-z.-]+|o[1-9][a-z0-9-]*|claude-[a-z0-9.-]+|gemini-[0-9a-z.-]+)\"" | sort | uniq -c | sort -rn
g "(system|instructions).{0,80}(datetime\.now|Date\.now|new Date\(|uuid|time\.time)"

What triggers it. The description lists phrases such as “our OpenAI/Anthropic bill spiked”, “prompt caching isn’t hitting”, “the agent looped and cost $X”, “route cheap queries to a smaller model” and “track AI spend by customer.”

How to install it. The skill is listed on skills.sh and installs with the open-source skills CLI:

# Preview what the repository contains
npx skills add build-with-ai-hq/llm-cost-reduction --list

# Install into the current project for Claude Code, Codex and Cursor
npx skills add build-with-ai-hq/llm-cost-reduction -a claude-code -a codex -a cursor

# Install globally (user level) and skip prompts
npx skills add build-with-ai-hq/llm-cost-reduction -g -y

# See installed skills, and pull updates later
npx skills list
npx skills update

In a test install on October 28, 2026, The Frontier Wire ran npx -y skills@1.7.4 add build-with-ai-hq/llm-cost-reduction -a claude-code -a codex -a cursor -y in an empty repository. The CLI reported “Installed 1 skill,” placed the files in .agents/skills/llm-cost-reduction (read directly by Codex and Cursor, symlinked for Claude Code), wrote a skills-lock.json with a computed hash, and printed: “Review skills before use; they run with full agent permissions.” SKILL.md runs to 2,724 words. The lint_skills.py linter that ships in the sibling agent-skills-for-teams skill reported no errors or warnings for it. The npx skills and skills.sh guide explains the CLI in full.

How to use it. Prompts that match its description include:

• “Our Anthropic bill spiked this month. Find which feature is burning tokens.”
• “Prompt caching isn’t hitting on our support bot. Find out why.”
• “Route cheap queries to a smaller model without breaking quality.”
• “Set a budget per team and track AI spend by customer.”

For the observability side of attribution, the sibling llm-observability-stack skill, covered in the LLM observability explainer, handles traces, dashboards and alerts.

Verifying savings and the limits of the playbook

A cost change is verified when cost per 1,000 requests fell for equal traffic windows and eval scores for every changed feature held. Totals mislead when traffic grows, so the skill compares normalized cost.

In practice: rerun the eval set for every feature whose model, prompt or context changed; confirm cached_tokens is above zero on repeated prefixes and response-cache hits are zero on personalized answers; force an agent into a loop in a test and confirm it stops; send a request on a key with a one-cent budget and confirm the refusal; and check p95 latency, since a classifier or embedding lookup adds time to every request.

The playbook has limits. It does not cover self-hosting open-weight models, fine-tuning or distillation, or contract negotiation, all of which change the price per token more than any prompt edit. Provider caching rules and prices change often; the skill’s caching reference covers both of Anthropic’s modes, automatic caching through a single top-level field and explicit breakpoints, and tells readers to check current provider docs before relying on numbers. The build-with-ai-hq GitHub organization is new, created on October 25, 2026, and its skills run with the agent’s full permissions; reading SKILL.md before installing takes a few minutes.

LLM cost optimization rewards discipline more than cleverness. The biggest savings are often in plain view once every call is attributed: a frontier model doing classification, a timestamp at the top of a system prompt, a nightly job paying synchronous prices. The llm-cost-reduction skill puts that checklist inside the coding agent that already has the repository open, and npx skills add build-with-ai-hq/llm-cost-reduction is enough to try it on one feature.