Should AI agent memory use a vector database or a graph database?

Use a vector database when your agent mostly needs to find text similar in meaning to the current request, a graph database when it needs to follow explicit relationships between entities, and both when it needs both, which is common for long-lived agent memory. Microsoft researchers' GraphRAG paper (Edge et al., 2024) draws the boundary concretely: conventional "vector RAG" works well for queries answered from "a small set of records", but not for questions that need an understanding of a whole dataset, where a graph of entities and relationships gave more comprehensive and diverse answers on two corpora of about 1 million tokens each.

How does each database find a memory?

A vector database finds memories by similarity; a graph database finds them by following links. In a vector store, each piece of text is turned into an embedding vector, and retrieval returns the stored vectors closest to the query's vector. The "Generative Agents" paper (Park et al., 2023) computes memory relevance this way, as cosine similarity between embeddings. In a graph store, facts are nodes and edges. GraphRAG uses an LLM to build a knowledge graph "where nodes correspond to key entities in the corpus and edges represent relationships between those entities", then groups closely related entities into communities and summarizes each one. Similarity tells you what sounds related. Edges tell you what is actually connected, such as who owns a service or which decision replaced another.

What does each approach handle well?

Each store is strong where the other is weak, based on how each one retrieves and on the GraphRAG authors' comparison with vector RAG.

Need	Vector database	Graph database
Find text similar in meaning to the request	Good fit: nearest-neighbor search over embeddings	Needs an added embedding index
Follow relationships (person, project, decision)	Weak: similarity does not store links	Good fit: relationships are explicit edges
Whole-dataset questions ("main themes")	Weak, according to GraphRAG	Better in GraphRAG's tests, via community summaries
Short, direct answers	In the GraphRAG tests on whole-dataset questions, vector RAG gave the most direct answers	Less direct in the same tests
Setup work	Chunk and embed the text	Extract entities and relationships, in GraphRAG with LLM calls per chunk
Record that one fact replaced another	Needs metadata you design	Can be stored as an edge

When is a vector database alone enough?

A vector database alone is enough when your agent answers questions over documents, serves one team in one tool, and the facts rarely change or depend on each other. Start there: it is the simpler system to run. Add a graph when you see one of three signals: questions about ownership, dependencies or timelines that similarity search answers badly; facts that change and need a record of what replaced what; or questions about a whole body of knowledge rather than one passage. Anthropic's advice to teams building agents applies here too: "do the simplest thing that works." For the difference between documents and memory, see agent memory vs RAG, and for picking the wider system, how to choose a memory layer.

How does elephant.broker combine the two?

elephant.broker uses both, behind a memory provider. The research paper on elephant.broker (arXiv 2603.25097) describes an architecture that "unifies a Neo4j knowledge graph with a Qdrant vector store through the Cognee SDK", and the product site says every turn "recalls from, and writes back to, one graph and vector store, gateway-isolated per tenant." The runtime is not itself a database: on every turn it retrieves from that store, reranks the results, builds a working set and passes it to your model, unchanged. Cognee is the default memory provider, and other memory providers can be plugged in. When two retrieved sources conflict, both are placed and weighed by proof; see what happens when agent memory conflicts.

Sources

• Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization (2024)
• Park et al., Generative Agents: Interactive Simulacra of Human Behavior (2023)
• Anthropic, Effective context engineering for AI agents (Sep 29, 2025)
• Lupascu and Lupascu, ElephantBroker: A Knowledge-Grounded Cognitive Runtime for Trustworthy AI Agents (2026)
• elephant.broker: how a turn flows, features

Product details come from each vendor's public documentation as viewed on 2026-10-28 and may have changed. Product names are trademarks of their respective owners. Cognee is the default memory provider in elephant.broker; elephant.broker is not affiliated with the other products named.