How can I reduce AI agent token costs without switching models?

You can cut AI agent token costs without switching models in two ways: pay less per token, and send fewer tokens per turn. On price, Anthropic's prompt caching documentation bills cache reads at 0.1 times the base input price on most Claude models (0.05 times on Opus 5.5 and Sonnet 5.5), OpenAI's guide quotes cached input "discounted up to 95%", and Anthropic's Batch API gives "a 50 percent discount on both input and output tokens" for asynchronous work. On volume, clearing old tool results, compacting long sessions, retrieving fewer documents and pruning each turn's working set reduce the tokens you pay for, with the same model.

Which cost levers keep the same model?

Most levers keep your model; only routing changes it. Ratios come from Anthropic's documentation, viewed 2026-10-28:

Approach	What it changes	What it costs you
Prompt caching	Price of a repeated prefix: reads at 0.1x base input on most Claude models, 0.05x on Opus 5.5 and Sonnet 5.5	Writes at 1.25x (5-minute) or 2x (1-hour); any prefix change misses
Batch processing	Price of non-urgent calls: 50% off	Results arrive asynchronously
Model routing (AI gateways and model routers)	Which model answers each call	It does switch models; needs evals per route
Transcript compression or summarization	Size of the history carried forward	A summary call; may lose "subtle but critical context" (Anthropic)
Tighter retrieval	Documents per call	Risk of missing a needed document
Pruning the working set per turn	What each turn sends	An extra selection step per turn

How much does prompt caching save?

Prompt caching saves the most when a long prefix repeats unchanged across many calls. OpenAI's prompt caching guide gives the arithmetic for GPT-5.6 and later models, with writes at 1.25 times and reads at 0.1 times the ordinary input rate on most of those models: across ten requests, "one write and nine full reads cost 2.15× at that rate, compared with 10× without caching." Caching has limits. The prefix must match exactly, so timestamps or per-user data near the top break it. Both providers set a minimum cacheable length: 1,024 tokens for GPT-5.6 and later, and 512 to 4,096 tokens on Claude models, depending on the model. And cached tokens still fill the window: Anthropic notes that caching "changes what you pay for those tokens, not whether they count," so it does nothing for context rot.

How do I cut the tokens each turn sends?

Find where the tokens go, then remove what the current step does not use:

1. Log input tokens per turn from each response's usage field, split into system prompt, tool definitions, history and tool results.
2. Put stable instructions and tool definitions first and changing content last, so the prefix stays cacheable.
3. Clear tool results the agent has already used.
4. Load only the tools the task needs; Anthropic lists bloated tool sets as "one of the most common failure modes."
5. Compact long sessions and check what the summary drops.
6. Retrieve fewer, better documents and rerank them before sending; Lost in the Middle found that going past 20 retrieved documents added only about 1.5% accuracy for GPT-3.5-Turbo.

For coding agents, see how to reduce Claude Code token usage.

Where does elephant.broker fit?

elephant.broker works on token volume, not price: on every turn the runtime retrieves, reranks and prunes context into a smaller working set, while your model and harness stay unchanged. elephant.broker reports on its website 40% lower AI spend with the same model (the site adds that "savings compound on long-running work") and 40-80% less context per task. These are elephant.broker's own reported results, not an independent or head-to-head test. The runtime adds a retrieve and rerank step to every turn. Spend dashboards in the runtime show what every agent, task and procedure costs, and let you cap it. If one person runs short sessions in one tool, caching and built-in compaction may be enough.

Sources

• Anthropic: Prompt caching, Pricing, Context windows, Effective context engineering for AI agents
• OpenAI: Prompt caching guide
• Liu et al.: Lost in the Middle: How Language Models Use Long Contexts, TACL 2023
• elephant.broker: overview, features

Product details come from each vendor's public documentation as viewed on 2026-10-28 and may have changed. Product names are trademarks of their respective owners; elephant.broker is not affiliated with them.