What is memory for AI agents?

Memory for AI agents is information an agent stores outside the language model and loads back into the context window on a later turn or session, because the model itself keeps nothing between calls. The Princeton paper "Cognitive Architectures for Language Agents" (CoALA, Sumers et al., 2023) puts it plainly: "Language models are stateless: they do not persist information across calls." CoALA divides agent memory into short-term working memory and three long-term types: episodic, semantic and procedural. In production, storing memory is the easy part. Deciding which memories enter the window on each turn is where agents succeed or fail.

What are the four types of agent memory?

CoALA names four memory types, and each answers a different question for the agent. The definitions below follow the paper; the examples show what each type looks like in a company agent.

Memory type	What CoALA says it holds	Example in a company agent
Working	Active information for the current decision cycle: inputs, active knowledge, goals	The open ticket and the last tool results
Episodic	"Experience from earlier decision cycles"	Last Friday's deploy failed at the migration step
Semantic	"Knowledge about the world and itself"	Refund rules, who owns which service
Procedural	Knowledge in the LLM weights plus the agent's code	Runbooks, tool definitions, approval steps

CoALA also warns that letting an agent rewrite its own procedural memory "is significantly riskier than writing to episodic or semantic memory", because it "can easily introduce bugs or allow an agent to subvert its designers' intentions."

How does an agent decide which memories to recall?

An agent recalls memory through a retrieval step that scores stored items against the current situation and loads only the top ones into the context window. In the Stanford and Google paper "Generative Agents" (Park et al., 2023), each memory gets three scores: recency (exponential decay since the memory was last retrieved), importance (an integer the model assigns when the memory is written, for example 2 for "cleaning up the room") and relevance (cosine similarity between embedding vectors). The top-ranked memories that fit the window go into the prompt. Anthropic's guide "Effective context engineering for AI agents" explains why the cut matters: as the number of tokens in the window grows, the model's ability to recall information from it decreases, so context "must be treated as a finite resource with diminishing marginal returns."

Do you need a separate memory system?

You may not: for one person working in one tool, the memory built into that tool often covers it. Claude Code loads CLAUDE.md instructions and its own auto memory notes at the start of every session, and Anthropic's memory tool lets Claude create, read, update and delete files in a /memories directory that your application stores. Both have clear edges. Claude Code's documentation says auto memory "is machine-local" and its files "are not shared across machines or cloud environments." A separate system starts to pay off when several agents, tools or teams need the same facts, or when stale notes start to outweigh useful ones. See how Claude Code memory works, agent memory vs RAG and how to choose a memory layer.

Where does elephant.broker fit?

elephant.broker is not a memory store but a pluggable runtime for AI agents that sits between your harness and your business and decides, on every turn, what your agent sees and what stays out. The runtime uses a memory provider, Cognee by default, and other providers can be plugged in. Before each turn, the runtime assembles context from your tools, teammates and past sessions, ranks it, drops stale or duplicate facts and places the working set where the model pays attention. After the turn, elephant.broker's post-turn hook captures decisions, corrections and outcomes and writes them back for the next turn. Your ways of working, procedural memory in CoALA's terms, become versioned procedures, placed for the task at hand. The cost is one more retrieve and rerank step on every turn; your model and harness stay unchanged. See also what context engineering is.

Sources

• Sumers, Yao, Narasimhan, Griffiths, Cognitive Architectures for Language Agents (2023)
• Park et al., Generative Agents: Interactive Simulacra of Human Behavior (2023)
• Anthropic, Effective context engineering for AI agents (Sep 29, 2025)
• Anthropic, Memory tool documentation
• Claude Code documentation, How Claude remembers your project
• elephant.broker: how a turn flows, features

Product details come from each vendor's public documentation as viewed on 2026-10-23 and may have changed. Product names are trademarks of their respective owners. Cognee is the default memory provider in elephant.broker; elephant.broker is not affiliated with the other products named.