Persisted HNSW. Format v9.
v0.16 embeds the prebuilt HNSW graph inside every .feather file — cold load is now 5–6× faster with deterministic recall. This page also summarises every major change since v0.10.
At a glance — v0.16
Added — persisted HNSW graph
File format v9 · saveIndexStream / loadIndexStream
save() now embeds the prebuilt HNSW graph inline in the .feather file. load() restores it directly — no rebuild step. On a 40k×128 dataset: 7.6 s serial → 1.7 s parallel → ~0.3 s with graph persistence.
- ·File grows ~25% (link lists stored). Load speedup justifies the trade-off at any meaningful dataset size.
- ·Deterministic recall: 0.988 vs 0.963 from parallel rebuild — reproducible results across restarts.
- ·Transparent fallback: rebuilds via existing parallel path when records have been forgotten/purged or modality uses on-disk quantization.
- ·v3–v8 files load unchanged. No migration step required.
import feather_db as fdb
db = fdb.DB("./agents.feather")
# ... add vectors ...
db.save() # graph embedded inline
db2 = fdb.DB("./agents.feather")
db2.load() # restores graph directly — no rebuildSince v0.10 — performance
v0.15.3 — Adaptive index capacity
HNSW indices grow on demand starting at 4,096 elements instead of preallocating 1M upfront. A 19-namespace workload dropped from 709 MB to 92 MB — 7.7× RAM reduction. compact() right-sizes to actual survivor count. No API or format changes.
v0.13.0 — Parallel load + batch ingestion + SIMD
- ·Parallel load — HNSW rebuilt in parallel across a thread pool. 4.5–4.8× faster (40k×128: 7.6 s → 1.7 s). Controlled via
FEATHER_LOAD_THREADS. - ·
DB.add_batch()— bulk-insert with parallel HNSW build and GIL release. 3.4× faster than serial (40k×128: 15.1 s → 4.5 s), recall 1.000. - ·SIMD kernels — hand-written SSE/AVX L2 distance compiled on x86_64. Runtime dispatch via
FEATHER_SIMD=none|sse|avx|avx512.
Since v0.10 — storage
v0.15.0 — In-RAM int8 quantization (format v8)
set_int8_ram(modality, max_abs) stores vectors as int8 in memory using global scale = max_abs/127. Result: 1.6–1.8× less RAM (227 MB → 129 MB on 60k×768) with recall ~0.88. Query/insert quantize transparently; get_vector dequantizes.
v0.12.0 — On-disk int8 quantization (format v7)
set_quantized(modality, on) persists vectors as int8 + per-vector scale. 2.5–4× smaller .feather files with max element error 0.39%. Per-vector scale = max|v|/127. Setting persisted across reloads.
Since v0.10 — search
v0.11.0 — Secondary indexes + pre-filtered ANN
Inverted indexes on namespace_id, entity_id, and each attribute key=value pair. Filtered lookups go from O(n) scan to O(matches).
- ·New methods:
ids_in_namespace(),ids_for_entity(),ids_with_attribute(),namespace_size(),list_namespaces() - ·Pre-filtered ANN:
search(..., filter=...)resolves candidates from secondary indexes when the filter targets indexed fields. Returns up to k matches whenever ≥k records match — fixes the "selective filter returns far fewer than k" bug from HNSW's ef-bounded traversal. - ·Auto-compaction:
set_auto_compact(ratio)triggers rebuild when deleted/total ratio crosses threshold.
Since v0.10 — MCP connector
v0.14.0 + v0.15.1 — Remote backend + real embedders
feather-serve now accepts --api-url and --api-key to proxy Claude (Desktop or Code) through to a deployed Feather Cloud API — no local file needed.
# connect Claude Desktop / Code to hosted Feather
feather-serve \
--api-url https://your-feather.example.com \
--api-key $FEATHER_API_KEY \
--embed-provider openai # real semantic search--embed-provider supports gemini, openai, voyage, cohere, ollama. Without it, the MCP connector used random vectors — semantic recall was non-functional.
Since v0.10 — Cloud API
Bulk delete
POST /v1/{ns}/records/batch_delete removes up to 100,000 records in a single namespace lock with one save() call — fixing the N-serialize problem of single-record deletes that could wedge the server under load.
POST /v1/{ns}/records/batch_delete
{
"ids": [1001, 1002, 1003], # optional — list of specific IDs
"entity_id": "creative_001", # optional — deletes all records for this entity
"cascade": true # default false — prunes graph edges in one sweep
}
# Response
{ "deleted": 3, "not_found": 0, "edges_pruned": 7 }Also available in the admin dashboard under Maintenance → Bulk delete. Supports ids, entity_id, or both (union). cascade: true automatically prunes all inbound and outbound edges referencing the deleted IDs.
Fast bulk import + flush
POST /v1/{ns}/import now uses throttled saves — the namespace is serialized at most once per FEATHER_IMPORT_SAVE_INTERVAL_S (default 30 s) instead of after every batch. Large imports that previously wedged the server now complete without I/O spikes.
A new POST /v1/{ns}/flush endpoint forces an immediate save + WAL compact — useful after a bulk import when you want durability before the throttle window expires.
# import a large dataset — saves are throttled automatically
POST /v1/{ns}/import
[{ "content": "...", "metadata": { "namespace_id": "acme" } }, ...]
# force durability immediately after
POST /v1/{ns}/flush
# → { "saved": true, "wal_compacted": true }ingest_text also benefits from throttled saves. Override the interval: FEATHER_IMPORT_SAVE_INTERVAL_S=10.
New REST endpoints
# Index introspection + maintenance (v0.13.1)
GET /v1/{ns}/admin/index_stats dim, record count, quantization state, threshold
PUT /v1/{ns}/admin/auto_compact enable incremental compaction { "ratio": 0.2 }
PUT /v1/{ns}/admin/quantize toggle on-disk int8 { "modality": "text", "on": true }
# Keyword search (v0.13.1)
POST /v1/{ns}/keyword_search BM25 only (no vector required)
# Upload local .feather to cloud (unreleased)
POST /v1/admin/upload stream local file → cloud namespacePer-namespace embedding dimensions (unreleased)
POST /v1/namespaces now accepts an optional dim parameter. Dimension constraints are enforced per-namespace — a mismatch returns a clear per-item error instead of silently padding or truncating.
POST /v1/namespaces
{ "name": "acme", "dim": 1536 } # locked to OpenAI 3-small dimsBug fixes (since v0.10)
Delete persistence
forget() and purge() now survive save+reload. Previously, forgotten vectors resurrected as orphans. C++ save_vectors() rewritten to exclude soft-deleted records.
Dangling edges on record delete
Deleting a record now strips inbound and outbound edges referencing it. Graph view no longer surfaces phantom nodes.
context_chain 500 error
/v1/{ns}/context_chain was reading e.source_id / e.target_id but the ContextEdge binding exposes .source / .target. Fixed in v0.14.0.
Graph endpoint crash on large namespaces
GET /v1/{ns}/graph now returns a structured error ("too large — filter by namespace_id/entity") instead of a 500 when the node set exceeds safe serialization bounds.
Where to go next
Try the Cloud Edition in 5 minutes
Spin up the Docker image and open http://localhost:8000/admin/.
git clone https://github.com/feather-store/feather.git
cd feather
docker compose -f feather-api/docker-compose.yml build
FEATHER_API_KEY="feather-$(openssl rand -hex 16)" \
docker compose -f feather-api/docker-compose.yml up -d