Everyone asks what a memory layer costs to run locally when you refuse to add a vector database. I measured it rather than guessing, and I couldn't find a comparable published table, so here it is. Method: synthetic entries of ~30 words each, Zipf-distributed 3,000-word vocabulary, measured with tracemalloc. Retrieval is BM25 running in-process, storage is a single SQLite file. entries resident RAM JSON on disk search 1,000 4.8 MB 1.7 MB 7 ms 5,000 23 MB 8.4 MB 34 ms 10,000 46 MB 17 MB 68 ms ~4.6 KB per entry in memory, ~1.7 KB on disk — both linear. Three things worth knowing: The vocabulary is not the cost. It saturates around 3,000 unique terms and stops growing. What costs you is the per-entry TF-IDF vector: a Python dict of term→float held for every entry whether or not it is ever queried. A compact postings-list index would be much smaller. Dict-per-document is easy to write and expensive to hold. Search is a linear scan, not an inverted index. It walks every cached vector and computes cosine similarity, so latency tracks N: 7 ms at 1k, 68 ms at 10k, ~170 ms at 25k. Fine for a few recalls per turn, wrong for a query-heavy workload — and no amount of tuning fixes the asymptote. Persistence rewrites the whole JSON file on every store, so insert cost grows with the store and the total is quadratic. First thing to change past a few thousand entries. Linear extrapolation: 25k ≈ 115 MB resident, 50k ≈ 230 MB, 100k ≈ 460 MB. The backend's default cap is 10,000, past which it evicts lowest-relevance entries instead of growing. So on a small box it's comfortable to about 10k entries, which covers a personal agent's history. Past that, embeddings or a real index is the right answer — BM25 in-process is a starting point, not a destination. What I'd genuinely like from this sub: the same table for an embedding-backed store. That's the comparison everyone actually wants and I only have half of it. If you've measured a local vector store's RAM per entry at these sizes, I'd like the numbers. Disclosure: I wrote this memory layer (CogniCore, MIT licensed, free). Happy to share the ~40-line benchmark script if anyone wants to reproduce or extend it — the numbers are reproducible. Repo: https://github.com/cognicore-dev/cognicore-env   submitted by   /u/Neither-Witness-6010 [link]   [comments]