What is context engineering for AI agents?

Context engineering is the practice of deciding which tokens an AI agent's model sees on each call, and Anthropic defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference." For an agent that works over many turns, that set includes system instructions, tools, Model Context Protocol (MCP) connections, external data and message history, and it changes on every turn. Anthropic's guiding principle is to find "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome."

How is context engineering different from prompt engineering?

Prompt engineering is about writing the instructions; context engineering is about everything that lands in the context window on each turn, the instructions included. Anthropic calls context engineering "the natural progression of prompt engineering." Writing a prompt is a one-time task, while context curation repeats before every model call. The difference matters for agents because, in Anthropic's words, an agent "running in a loop generates more and more data that could be relevant for the next turn of inference." Tool outputs, notes, retrieved documents and old messages all compete with your instructions for the same window, so someone, or something, has to decide what stays in.

Why does context engineering matter for AI agents?

Context engineering matters because model accuracy falls as the context grows, even when the right facts are present. A July 2025 Chroma technical report by Hong, Troynikov and Huber, "Context Rot: How Increasing Input Tokens Impacts LLM Performance", tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, and found that "model performance consistently degrades with increasing input length" while the task stayed the same. Anthropic's engineering team explains the mechanism as an "attention budget": in a transformer every token attends to every other token, which gives n² pairwise relationships for n tokens, and every new token "depletes this budget by some amount." More on this in what context rot is and whether a bigger context window helps.

What techniques does context engineering use?

Anthropic's context engineering guide covers these techniques:

Technique	What it does	Fit or trade-off, per Anthropic
Just-in-time retrieval	Keeps "lightweight identifiers" (file paths, queries, links) and loads data with tools when needed	Slower than pre-computed retrieval; Claude Code mixes both
Compaction	Summarizes a conversation near the window limit and restarts from the summary	Long back-and-forth tasks; aggressive compaction can lose "subtle but critical context"
Tool result clearing	Drops raw tool outputs from deep in the history	"One of the safest lightest touch forms of compaction"
Structured note-taking	The agent writes notes outside the window and reads them back later	Iterative work with clear milestones
Sub-agents	Each works in a clean window and returns a summary, "often 1,000-2,000 tokens"	Complex research with parallel exploration

Where does elephant.broker fit?

elephant.broker is a pluggable runtime for AI agents that does context engineering on every turn, between your harness and the model call. The runtime assembles task-relevant context from your tools, teammates and past sessions, ranks and prunes it, drops stale or duplicate facts, places the working set where the model pays attention, and writes the outcome back for the next turn. Your model and harness stay unchanged, and the runtime connects through an MCP server, native plugins or a REST API. elephant.broker reports on its website 40-80% less context per task, with "only verified, relevant knowledge" reaching the window; that is elephant.broker's own reported result, not an independent or head-to-head test. The trade-off is an extra retrieve and rerank step on every turn. For one person running short sessions in one tool, the harness's built-in compaction may be enough.

Sources

• Anthropic: Effective context engineering for AI agents, published 29 September 2025
• Chroma (Hong, Troynikov and Huber): Context Rot: How Increasing Input Tokens Impacts LLM Performance, technical report, 14 July 2025
• elephant.broker: overview, how a turn flows, research paper, arXiv 2603.25097

Product details come from each vendor's public documentation as viewed on 2026-10-28 and may have changed. Product names are trademarks of their respective owners; elephant.broker is not affiliated with them.