How to Build a Context Engine in Python: Complete Tutorial
A context engine retrieves the most relevant past information before each LLM call, replacing expensive full-context stuffing with targeted semantic search. This tutorial builds one end-to-end using Feather DB in under 50 lines of Python.
A context engine sits between your application and your LLM: when a new query arrives, it retrieves the most relevant past information, injects it into the prompt, and discards the rest. This lets you replace a 57,000-token context window (costing $0.29/session) with a targeted 1,500-token context block ($0.008/session) while improving recall accuracy from 0.640 to 0.693 on LongMemEval. This tutorial builds a production-ready context engine in Python using Feather DB, covering installation, ingestion, decay configuration, semantic search, and graph traversal.
What a Context Engine Is (and Is Not)
A context engine is not a RAG pipeline. RAG retrieves documents. A context engine retrieves memories — accumulated, adaptive, decaying knowledge about a user, a project, or a domain. The difference is in what gets stored:
- RAG: Static documents. Retrieves relevant passages from a fixed corpus.
- Context engine: Dynamic memories. Retrieves the most relevant facts from an evolving, weighted, time-decaying knowledge store.
Feather DB is designed specifically for context engines — the adaptive decay, stickiness model, and graph traversal exist because context engines need different retrieval semantics than document search.
Step 1: Install Feather DB
pip install feather-db
That is the entire install. No server, no Docker, no infrastructure. Feather DB is an embedded C++ library with Python bindings — it runs in your Python process.
Step 2: Initialize the Database
import feather_db as fdb
# Create or open a context engine
# The .feather file is created automatically if it doesn't exist
db = fdb.FeatherDB("context_engine.feather")
print(f"Engine initialized. Nodes: {db.count()}")
Step 3: Ingest Data with Decay Settings
Every piece of information you store has three parameters that control how it ages:
importance(0.0–1.0): How much this fact matters. High importance = slower decay.half_life_days: How many days before this memory is half as likely to be retrieved. Default: 60.metadata: Arbitrary key-value pairs for filtering.
import feather_db as fdb
from datetime import datetime
db = fdb.FeatherDB("context_engine.feather")
# Ingest a high-importance, durable fact
db.add(
text="User's primary tech stack is Python 3.11 + FastAPI + PostgreSQL",
metadata={
"entity": "user_001",
"category": "technical_profile",
"session_id": "onboarding",
"timestamp": datetime.now().isoformat()
},
importance=0.95,
half_life_days=180 # Durable — tech stack changes slowly
)
# Ingest a medium-importance, faster-decaying memory
db.add(
text="User is debugging a memory leak in their FastAPI app — high urgency",
metadata={
"entity": "user_001",
"category": "active_task",
"session_id": "session_042",
"timestamp": datetime.now().isoformat()
},
importance=0.7,
half_life_days=7 # Ephemeral — active tasks resolve quickly
)
# Ingest a low-importance contextual note
db.add(
text="User mentioned they prefer dark mode in their IDE",
metadata={
"entity": "user_001",
"category": "preference",
"session_id": "session_015",
"timestamp": datetime.now().isoformat()
},
importance=0.4,
half_life_days=365
)
print(f"Total nodes: {db.count()}")
Step 4: Batch Ingestion for Existing Data
For loading existing conversation history or knowledge bases, use batch ingestion:
import feather_db as fdb
db = fdb.FeatherDB("context_engine.feather")
# Batch add from a list of facts
facts = [
{
"text": "User's company uses Kubernetes on AWS EKS",
"importance": 0.85,
"half_life_days": 365,
"metadata": {"entity": "user_001", "category": "infrastructure"}
},
{
"text": "User reported slow cold starts on Lambda functions — investigating",
"importance": 0.65,
"half_life_days": 14,
"metadata": {"entity": "user_001", "category": "active_task"}
},
{
"text": "User's team has 6 engineers, 2 on backend",
"importance": 0.6,
"half_life_days": 180,
"metadata": {"entity": "user_001", "category": "team"}
}
]
for fact in facts:
db.add(
text=fact["text"],
metadata=fact["metadata"],
importance=fact["importance"],
half_life_days=fact["half_life_days"]
)
print(f"Ingested {len(facts)} facts. Total: {db.count()}")
Step 5: Configure Decay Parameters
Decay in Feather DB is governed by the adaptive scoring formula:
# How Feather DB scores each memory at query time:
# stickiness = 1 + log(1 + recall_count) # Frequently recalled memories resist aging
# effective_age = age_in_days / stickiness
# recency = 0.5 ^ (effective_age / half_life_days)
# final_score = ((1 - time_weight) * similarity + time_weight * recency) * importance
# You can tune time_weight per query:
results = db.search(
query="What is the user's tech stack?",
top_k=5,
filters={"entity": "user_001"},
time_weight=0.3 # 0 = pure similarity, 1 = pure recency
)
A time_weight of 0.3 means 70% semantic similarity and 30% recency. For agent memory, 0.2–0.4 is the practical range. Higher values favor recent memories even if less relevant; lower values favor semantic match regardless of age.
Step 6: Query with Semantic Search
import feather_db as fdb
db = fdb.FeatherDB("context_engine.feather")
# Basic semantic search
results = db.search(
query="What is the user working on right now?",
top_k=8,
filters={"entity": "user_001"}
)
for r in results:
print(f"Score: {r['score']:.4f} | {r['text'][:80]}")
The search uses HNSW for approximate nearest neighbor + BM25 for exact-term matching, fused via Reciprocal Rank Fusion. You get hybrid retrieval automatically — semantic for meaning, lexical for exact terms like function names, error codes, or identifiers.
Step 7: Traverse Context Graph
After adding memories, you can link them to surface relationships that vector similarity alone would miss:
import feather_db as fdb
db = fdb.FeatherDB("context_engine.feather")
# Add related memories and link them
bug_id = db.add(
text="User reported memory leak in payment handler (session_042)",
metadata={"entity": "user_001", "category": "bug"},
importance=0.8
)
fix_id = db.add(
text="Fixed memory leak: connection pool not being released in payment_handler.py:line 247",
metadata={"entity": "user_001", "category": "fix"},
importance=0.9
)
# Link the bug to the fix
db.link_nodes(
source_id=bug_id,
target_id=fix_id,
edge_type="leads_to",
weight=1.0
)
# Now retrieve with graph traversal
# Starting from ANN results, traverse 2 hops of edges
results = db.context_chain(
query="payment handler bug",
hops=2,
filters={"entity": "user_001"},
edge_types=["leads_to", "refines", "supersedes"]
)
for r in results:
print(f"[{r.get('edge_type', 'direct')}] {r['text'][:80]}")
Step 8: Build the Context Block for LLM Injection
The final step is formatting retrieved memories into a context block for your LLM:
import feather_db as fdb
from openai import OpenAI
db = fdb.FeatherDB("context_engine.feather")
client = OpenAI()
def get_context_block(user_id: str, query: str, top_k: int = 8) -> str:
"""Retrieve and format the most relevant memories as a context block."""
memories = db.context_chain(
query=query,
hops=2,
filters={"entity": user_id},
top_k=top_k
)
if not memories:
return "No relevant context found."
lines = [f"- {m['text']}" for m in memories]
return "\n".join(lines)
def chat_with_context(user_id: str, user_message: str) -> str:
"""Send a message to the LLM with retrieved context."""
context = get_context_block(user_id, user_message)
system_prompt = f"""You are a helpful assistant with memory of past interactions.
Relevant context from memory:
{context}
Use this context to give personalized, informed responses."""
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message}
]
)
return response.choices[0].message.content
# Usage
reply = chat_with_context("user_001", "Help me debug the memory leak issue")
print(reply)
Cost Calculation
Here is the cost difference this context engine produces at GPT-4o pricing ($5/1M input tokens):
# Without context engine: stuff all history
tokens_without_engine = 57_000 # ~6 months of conversation history
cost_without = (tokens_without_engine / 1_000_000) * 5 # $0.285 per session
# With context engine: inject top 8 memories
tokens_with_engine = 1_500 # ~8 memories @ ~185 tokens each
cost_with = (tokens_with_engine / 1_000_000) * 5 # $0.0075 per session
savings_per_1000_sessions = (cost_without - cost_with) * 1000
print(f"Cost without engine: ${cost_without:.4f}/session")
print(f"Cost with engine: ${cost_with:.4f}/session")
print(f"Savings per 1,000 sessions: ${savings_per_1000_sessions:.2f}")
# Output:
# Cost without engine: $0.2850/session
# Cost with engine: $0.0075/session
# Savings per 1,000 sessions: $277.50
Production Considerations
File persistence
The .feather file persists automatically between runs. Mount it on a persistent volume in Docker or Kubernetes, or include it in your application's data directory. The file is portable — copy it between environments without rebuilding the index.
Multi-user isolation
Use one .feather file per user (recommended for clear isolation), or use a single file with namespace/entity filtering. The filters={"entity": user_id} parameter on every query ensures user data never crosses boundaries.
Index rebuild on startup
Feather DB v0.16 cold loads 5–6x faster than previous versions. A 100K-node index loads in under a second on modern hardware. No rebuild is needed — the HNSW index is persisted in the .feather file.
FAQ
How is a context engine different from RAG?
RAG retrieves passages from static documents. A context engine retrieves dynamic, time-weighted memories that decay, accumulate stickiness from repeated recall, and link to each other via typed edges. The retrieval semantics are designed for agent memory, not document search.
How many memories should I store per user?
Feather DB handles 500K+ nodes at 0.19ms p50. Practically, most agent applications accumulate 1,000–50,000 memories per user over a year. At this scale, performance is identical to a 100-node index.
Do I need to embed the text myself?
Feather DB handles embedding internally when you pass text to db.add() and db.search(). You can also pass pre-computed embeddings if you want to use a specific model or cache embeddings externally.
What embedding model does Feather DB use by default?
Feather DB uses a configurable embedding model. The default is a lightweight local model for zero-dependency installs; you can configure it to use OpenAI, Cohere, or any HuggingFace-compatible model.
How do I delete memories?
Use db.delete(node_id) for individual nodes or db.delete_by_filter({"entity": user_id}) to delete all memories for a user. Deletion is immediate and permanent.