Agent Memory Architectures: Vector DBs vs Knowledge Graphs vs Long-Term Store

Agent Memory Architectures: Vector DBs vs Knowledge Graphs vs Long-Term Store

Reviewed: June 4, 2026

Memory is what separates a chatbot from an agent. Without memory, every interaction starts from scratch — no continuity, no personalization, no accumulated knowledge. As agents tackle longer and more complex tasks, memory architecture becomes the most consequential design decision. This post breaks down the three dominant approaches and shows you when to use each.

The Three Memory Tiers

Agent memory operates across three time horizons:

• Working memory (in-context): What the agent can see right now — the current conversation, retrieved documents, and tool outputs. Limited by the LLM context window.
• Short-term store (session-scoped): Information accumulated during a session — decisions made, plans formulated, intermediate results. Lost when the session ends unless explicitly persisted.
• Long-term store (persistent): Knowledge that survives across sessions — user preferences, domain facts, past interactions, learned skills. This is where architecture choices matter most.

Architecture 1: Vector Database Retrieval

The most common long-term memory architecture. Conversations, documents, and facts are embedded as vectors and stored in a vector database. At query time, the agent retrieves semantically similar memories.

class VectorMemoryStore:
    def __init__(self):
        self.embedder = OpenAIEmbeddings()
        self.db = ChromaDB(persistent_path="./agent_memory")
    
    def remember(self, text, metadata=None):
        vector = self.embedder.embed(text)
        self.db.add(vector, text, metadata or {})
    
    def recall(self, query, top_k=5):
        query_vec = self.embedder.embed(query)
        return self.db.search(query_vec, k=top_k)
    
    def reflect(self, last_n=20):
        """Consolidate recent memories to reduce redundancy"""
        recent = self.db.get_last(n=last_n)
        summary = llm.summarize(recent)
        self.remember(summary, {"type": "consolidated"})

Strengths:

• Simple to implement, excellent semantic matching
• Scales to millions of memories
• Mature ecosystem (Pinecone, Weaviate, Chroma, Qdrant)

Weaknesses:

• No understanding of relationships between memories
• Retrieval quality degrades with ambiguous queries
• No temporal reasoning — „what changed last week?“ is hard
• Redundant storage of related facts

Architecture 2: Knowledge Graphs

Store memories as entities and relationships. The agent can traverse the graph to find indirect connections, reason about relationships, and maintain ontological structure.

class KnowledgeGraphMemory:
    def remember(self, facts):
        """facts = [{'subject': 'Alice', 'predicate': 'works_at', 'object': 'Google'}, ...]"""
        for fact in facts:
            self.graph.add_triple(fact['subject'], fact['predicate'], fact['object'])
    
    def recall(self, entity, depth=2):
        """Find everything connected to an entity within N hops"""
        return self.graph.traverse(entity, max_depth=depth)
    
    def infer(self, query):
        """Apply graph reasoning rules"""
        return self.reasoner.apply_rules(query, self.graph)

Strengths:

• Rich relationship modeling — „who does Alice report to?“
• Inferencing over transitive relationships
• Explainable reasoning paths
• Efficient storage of known facts (no duplication)

Weaknesses:

class HybridAgentMemory:
    def __init__(self):
        self.vector_store = VectorMemoryStore()      # Semantic recall
        self.kg = KnowledgeGraphMemory()               # Relationship reasoning
        self.kv = KeyValueStore()                      # Fast lookups (user prefs, state)
    
    def remember(self, text, structured_facts=None):
        self.vector_store.remember(text)
        if structured_facts:
            self.kg.remember(structured_facts)
    
    def recall(self, query):
        # Parallel retrieval across all stores
        semantic_results = self.vector_store.recall(query)
        graph_results = self.kg.recall(query)
        kv_results = self.kv.get(query)
        
        # Merge and rank
        return self.fusion_ranker.merge(semantic_results, graph_results, kv_results)