Agent Memory in the Context Era: The Architecture Behind AI That Remembers

Why AI agents need more than one type of memory

Cognitive science has long distinguished between different types of human memory: episodic (specific past events), semantic (general facts and concepts), procedural (how to do things), and working (what is currently active in consciousness). These distinctions are useful because each type has different retrieval characteristics, different decay rates, and different relevance to current behavior.

AI agents in the Context Era face the same structural challenge. A production agent needs to remember specific past interactions (episodic), accumulated facts about its domain and users (semantic), established patterns and workflows it has developed (procedural), and the active context of the current session (working). Treating all four as the same type of retrieval problem produces retrieval quality failures — episodic memories competing with semantic facts, procedural patterns surfacing when episodic recall is needed.

Feather DB implements all four memory types through a combination of metadata differentiation, decay parameter tuning, importance weighting, and graph relationships. The result is a context engine that retrieves the right type of memory for the current context — at 0.19ms p50 on 500K vectors with 97.2% recall@10.

The four memory types and how they work

Episodic memory stores specific past events: what happened in session 14, what the user said on March 3rd, what the agent recommended last Tuesday. Episodic memories are highly specific, time-stamped, and important for continuity across sessions. They decay at medium rate — too fast decay loses useful session history; too slow decay makes old irrelevant events compete with recent ones.

Semantic memory stores general facts about the world, the user, the domain, or the agent's operating context: the user's preferred programming language, the brand's target audience, the company's pricing structure. Semantic memories are stable across time — they should decay slowly and resist replacement unless explicitly superseded.

Procedural memory stores established patterns and workflows: how the agent handles a specific type of request, which sequence of steps resolves a class of issues, what creative approach works for a specific audience profile. Procedural memories are accumulated over many interactions and become more valuable with tenure — they should have high stickiness and slow decay.

Working memory holds the active context of the current session: what was just discussed, the current user intent, the intermediate state of a multi-step task. Working memory is ephemeral — it should not persist beyond the current session or, if it does, should decay rapidly.

Memory type comparison

Implementing all four memory types with Feather DB

import feather_db as fdb
import uuid
from your_embedder import embed

db = fdb.DB.open("agent_memory.feather", dim=768)

def store_memory(
    text: str,
    entity_id: str,
    memory_type: str,  # "episodic", "semantic", "procedural", "working"
    importance: float = 0.75
) -> str:
    meta = fdb.Metadata(importance=importance)
    meta.set_attribute("entity_id", entity_id)
    meta.set_attribute("memory_type", memory_type)
    meta.set_attribute("text", text)
    mem_id = str(uuid.uuid4())
    db.add(id=mem_id, vec=embed(text), meta=meta)
    return mem_id

# Decay parameters tuned by memory type
DECAY_PARAMS = {
    "episodic":   {"half_life": 30,  "time_weight": 0.3},
    "semantic":   {"half_life": 120, "time_weight": 0.15},
    "procedural": {"half_life": 150, "time_weight": 0.1},
    "working":    {"half_life": 3,   "time_weight": 0.6},
}

def retrieve_by_type(
    query: str,
    entity_id: str,
    memory_type: str,
    k: int = 8
) -> list[str]:
    params = DECAY_PARAMS[memory_type]
    results = db.search(
        embed(query),
        k=k,
        half_life=params["half_life"],
        time_weight=params["time_weight"],
        filters={"entity_id": entity_id, "memory_type": memory_type}
    )
    return [r.payload["text"] for r in results]

def retrieve_all_types(query: str, entity_id: str) -> dict:
    """Retrieve from all four memory types for a full context window."""
    return {
        "episodic":   retrieve_by_type(query, entity_id, "episodic", k=4),
        "semantic":   retrieve_by_type(query, entity_id, "semantic", k=5),
        "procedural": retrieve_by_type(query, entity_id, "procedural", k=3),
        "working":    retrieve_by_type(query, entity_id, "working", k=3),
    }

The retrieve_all_types function retrieves from each memory type with independently tuned decay parameters. Semantic memories retrieve with a 120-day half-life and 15% time weight — they are stable and should surface primarily on semantic relevance. Working memories retrieve with a 3-day half-life and 60% time weight — they are ephemeral and should surface primarily on recency.

Graph edges for causal memory chains

The graph layer connects memories across types in ways that flat retrieval cannot capture. A semantic fact ("user prefers TypeScript") can be linked to the episodic memory where that preference was established (session 3, where the user explicitly stated it) and to the procedural pattern it informed (always suggest TypeScript types for function parameters in this codebase).

When the agent retrieves the semantic fact at query time, BFS traversal can surface both the episodic origin (when and how the preference was established) and the procedural downstream (how it has been applied). The context window contains not just the fact but its causal history — which makes the agent's grounding more robust and its responses more coherent.

# Establish a causal memory chain
semantic_id = store_memory(
    text="User prefers TypeScript for all API handler functions.",
    entity_id=user_id,
    memory_type="semantic",
    importance=0.9
)
episodic_id = store_memory(
    text="Session 3: User explicitly said to always use TypeScript, "
         "not JavaScript, for API handlers. They had a bad experience with "
         "runtime type errors in a prior project.",
    entity_id=user_id,
    memory_type="episodic",
    importance=0.85
)
# Link: the semantic fact was established in this episodic event
db.link(semantic_id, episodic_id, rel_type="derived_from", weight=0.9)

The memory architecture behind LongMemEval 0.693

LongMemEval tests five capabilities: single-fact recall, multi-fact synthesis, temporal reasoning (before/after), preference tracking with updates, and knowledge correction when prior facts are superseded. Each capability maps to a different memory type and a different retrieval challenge:

Single-fact recall: semantic retrieval with high decay resistance. Multi-fact synthesis: cross-type retrieval plus graph traversal. Temporal reasoning: episodic retrieval with strong recency weighting. Preference tracking: semantic retrieval with supersedes edges for updates. Knowledge correction: supersedes graph edges that demote old facts without deletion.

Feather DB's 0.693 on LongMemEval with GPT-4o — vs. 0.640 for GPT-4o full-context — reflects this architecture working across all five capability types. The same LLM, given context from a properly structured multi-type memory system, performs better than when given a raw full-history dump. The gap comes from retrieval quality, not model quality.

Frequently Asked Questions

What are the four types of agent memory?

Episodic memory stores specific past events and session history. Semantic memory stores general facts, user preferences, and domain knowledge. Procedural memory stores established patterns and workflows accumulated through experience. Working memory holds the active context of the current session. Each type has different decay characteristics and retrieval relevance — treating all four as identical produces retrieval quality failures in production agents.

How does Feather DB implement working memory differently from semantic memory?

Working memory uses aggressive decay parameters: short half-life (1–7 days) and high time weight (0.6), ensuring that working memory is primarily ranked by recency. Sessions more than a week old decay out of active retrieval rapidly. Semantic memory uses conservative decay: long half-life (60–180 days) and low time weight (0.15), ensuring stability. Frequently recalled semantic memories acquire high stickiness and resist decay almost indefinitely.

How do supersedes edges handle knowledge updates?

When a fact changes — a user preference is updated, a prior recommendation is invalidated — a new semantic memory is stored with a supersedes edge to the old one. The old memory is not deleted; it retains its semantic embedding and is retrievable in explicit searches. But in relevance-ranked retrieval, the new memory scores higher (it is more recent, and the supersedes relationship demotes the old one in graph traversal). The agent naturally uses the updated fact without requiring manual deletion of the old one.

What is the LongMemEval benchmark and what does 0.693 mean?

LongMemEval is the standard benchmark for AI long-term memory, testing recall, temporal reasoning, preference tracking, and knowledge updates across simulated long-running agent conversations. A score of 0.693 means Feather DB with GPT-4o answers approximately 69.3% of long-term memory questions correctly. Full-context GPT-4o (the best possible LLM Era approach) scores 0.640. The 5.3 percentage point improvement reflects better context quality from the multi-type memory architecture.

How many memories can a production agent store per user?

There is no architectural limit — Feather DB scales to 500K+ vectors at 0.19ms p50 retrieval. For per-user stores, practical limits are set by decay: low-importance, rarely-recalled memories decay out of effective retrieval within weeks, keeping the active store size bounded even as total stored memory grows. A user with two years of interaction history might have 10,000 stored memories but an effective active set of 500–2,000 that surfaces in typical retrieval.