Hindsight Memlake

Agent memory on object storage

A storage engine written in Rust, built for memory that only grows. 100 million memories at under 100 ms p99, every write durable before it is acknowledged, and four kinds of search over one copy, in a bucket you control. Part of Hindsight Enterprise.

Get the whitepaper Talk to sales

Read the launch post →

recall p99 on a 3M-memory bank: 90 ms
memories on 4 servers, latency flat from 1 to 200 recalls/s: 100M
the ingest throughput of Postgres, per bank: 15–27×
how recall cost grows with bank size: √N

The workload

Agent memory only grows

Agents improve by remembering what worked and what didn’t, and unlike people, they never stop writing it down. Every fact, observation and mental model is more memory, and each one has to be findable in under a second.

An employee's agent writes about 68,000 memories a year, an autonomous agent about 5 million, and a 10,000-person company with 1,000 autonomous agents about 5.5 billion. Memory only grows, year after year. — Modelled: employees average 10 conversations a workday at ~27 memories each (measured); autonomous agents run ~500 tasks a day.

Where a relational store stops keeping up

Open-source Hindsight stores memory in Postgres, and for most deployments that’s the right choice. Running Hindsight across thousands of organizations, we measured where it stops keeping up with this workload:

Recall slows as a bank grows: On a 4-vCPU primary, recall served 7.0 queries per second at 33k memories and 1.9 at 164k.
Keyword search scores every match: On a 210k-memory bank, one query matched almost every row and took about 5 seconds.
Writes hit the write-ahead log: Each memory produced about 190 KB of WAL across its indexes, capping ingest at 55–80 memories a second.
Many banks hurt too: One organization with 1,000 small banks sustained only about 2 recalls per second.

Postgres isn’t doing anything wrong. The workload has an unusual shape: tiny records that only accumulate, four kinds of search over the same data on every recall, and every write visible the moment it lands. So we built a store for that shape of data.

Architecture

Four moving parts, and one of them is your bucket

The bucket

Each organization gets its own bucket, and each memory bank is a prefix inside it. Every write becomes one new, immutable log object; indexers fold the log into segments that are written once and never modified.

Conditional writes are the coordination

Only two objects ever change, the manifest and the head pointer, and only by conditional write: the store accepts a new version only if nobody changed it first. No database, lock service or leader beside the bucket.

Readers see whole versions

Because nothing is overwritten in place, a reader always sees a complete version of a bank, never something in between, and servers never need to talk to each other.

Writes

A server writes each retain to the log with a conditional create, so two writers can never claim the same position, then advances the head pointer with another conditional write, and only then acknowledges. A retain is a single log entry, so it lands entirely or not at all. Writes don’t wait for indexing: the next recall, from any server, reads the part of the log that hasn’t been indexed yet.

Recall

Each bank is routed to the same server so its cache, in memory and on local NVMe, stays warm, and since objects never change, that cache never needs invalidating. A recall checks the head pointer, reads only the blocks it needs, scans the unindexed tail of the log and runs four searches. Hindsight sends its ranking policy with the request, so the server ranks next to the data.

Failover

A bank’s index lives on one server at a time, and Memlake grows by spreading banks across servers. When a server disappears, another picks up its banks with no handover and warms its cache as it serves. It held a cache, never data.

Indexing

Indexers run apart from the servers that answer queries. An indexer folds new log entries into a fresh segment and publishes it by swapping the manifest, so indexing cost follows what’s new, not what’s stored. Two indexers can work on the same bank with no lock: one wins the swap, and the other’s files are garbage-collected.

Backups

An archiver copies every bank’s log, plus daily snapshots, to a separate bucket, ideally in another account with credentials that can’t delete. A bank can be restored to any point in time, and every restore is verified before traffic moves to it.

Written in Rust

Built from the ground up in Rust

Hindsight’s memory isn’t just vectors: entities, links and time are part of every recall. No off-the-shelf engine fit that shape, so we wrote one, in Rust, for the performance and the control over memory that a query path measured in milliseconds needs.

No garbage-collection pauses: Native code with no garbage collector, so tail latency isn’t at the mercy of a GC pause.
Memory budgets you can enforce: Caches and unindexed reads run on fixed budgets. Memlake pushes back on writers before it grows.
Search next to the data: Vector, BM25 (tantivy), graph and time search run in-process on the server that holds the bank’s cache, and rank before anything goes back over the wire.
Small, provable core: Servers, indexers and the archiver are Rust services over gRPC, coordinated only by conditional writes, and checked against the TLA+ specifications.

Design goals

Written down before the first line of code

Every change to Memlake is still reviewed against them.

Object storage is the only source of truth: No database, consensus cluster or second stateful system beside the bucket. Durability, replication and availability come from S3, GCS or Azure Blob Storage.
Durable before acknowledged: When Hindsight confirms a memory is saved, it is already in object storage, and the next recall from any server finds it.
Everything is incremental: A retain costs the same in a bank of a thousand memories or a hundred million. Indexing works on new data only.
One copy, four kinds of search: Each memory is stored once and serves vector, keyword, graph and time search. No separate vector database, search engine or graph store to keep in sync.
Compute is disposable: Servers hold only a cache. Losing one costs latency, never data, and capacity grows by adding servers.
Bounded by design: Caches run on fixed memory budgets. When writes outrun indexing, Memlake slows writers with retryable errors instead of growing memory.

Performance

Latency that barely moves as banks grow

Head to head through the full Hindsight API, same cluster, same data: on a small bank Memlake answers in a fifth of the time and serves ten times the traffic. On a large bank the gap is two orders of magnitude.

As one bank grows from 4k to 3M memories, Memlake stays flat at about 90 ms p99 and $1.7k a month. Postgres recall p99 rises from 0.5 s to 930 ms at 50k and 5.3 s at 210k, and its monthly cost grows to $7.4k at 210k and an extrapolated $73.3k at 3M. — One bank at 5 recalls/s. Recall p99 measured through the Hindsight API; cost modelled after a year, excluding the LLM and GPUs. Postgres beyond 210k memories is extrapolated.

Recall throughput: Memlake 207, 106, 113 and 113 recalls per second against Postgres 19, 0.45, 4 and 2 across four workloads. One recall at p99: Memlake 106 and 113 ms against Postgres 513 ms and 5.3 s. — Same cluster, same data, same benchmark driver, through the Hindsight API, September 2026, without the cross-encoder reranker. Memlake's 113 recalls/s is a ceiling of the test setup, not of Memlake.

Scales out across banks: 100 million memories across 20 banks ran on 4 servers of 6 vCPU each, about a year of agents for 1,500 employees. Latency stayed flat from 1 to 200 recalls per second.
Grows with the square root: Growing one bank 30×, from 100k to 3M memories, made a recall about 6× slower under heavy load. The largest single bank we’ve run holds 10 million memories.
Writes scale too: A single bank sustains 1,200–1,500 memories per second, flat as the bank grows: 15–27× what Postgres sustained.
No cost to retrieval quality: On BEIR datasets with the same embeddings, Memlake’s vector, keyword and hybrid search came within 0.3% of an established vector database on every comparison.

Recall latency: one 3M-memory bank, one request at a time, p50 71 ms and p99 90 ms; 100M memories on 4 servers, p50 40–43 ms and p99 73–80 ms from 1 to 200 recalls per second. — Engine-level benchmarks, July–August 2026. The 100M fleet ran on synthetic data; the 3M result on a realistic corpus.

SciFact and NFCorpus, nDCG@10, the same bge-small-en-v1.5 embeddings for both systems.

Correctness

Small enough to prove

Because correctness comes down to a handful of conditional writes, the write, indexing and garbage-collection protocols are specified in TLA+. A model checker explores every interleaving: many writers and indexers racing, servers crashing at any step, responses lost, processes stalling.

The specifications are re-checked in CI on every protocol change, and recorded executions of the real engine are checked against them so the code can’t drift from what was proven.

An acknowledged write is never lost, and a position in the log is never reused.
Every query sees every acknowledged write, from any server.
Readers always see a complete index, never half of an update.
Garbage collection never deletes anything still in use.
Two indexers racing on the same bank stay safe with no lock between them.

Cost and tradeoffs

Where it pays off, and where it doesn’t

A year for a whole company, one bank per employee or agent, each cluster sized to answer recall within 1 second at p99. Almost all of the difference is the memory store itself.

Monthly cluster cost after a year, excluding the LLM and GPUs: 1,000 employees (68M memories) $3.2k on Memlake against $4.6k on Postgres; 10,000 employees (680M) $15.1k against $58.4k; 100 autonomous agents (490M) $11.1k against $146k; 1,000 autonomous agents (4.9B) $94k against $2.46M. One recall at p99 is 0.35 s on Memlake in all four, against 2.4 to 11.1 s on Postgres. — Modelled, not billed: a full deployment at GCP list prices after 12 months of growth, excluding the LLM and the embedding and reranker GPUs, which cost the same on either store.

Storage is cheap; serving recall is the bill: About 4–5 KB per memory in the bucket, so a year of one employee’s agent is roughly 350 MB. On our hosted fleet, the storage tier is 2.2× cheaper on Memlake and the whole deployment 1.5×.
Small banks are fine on Postgres: Small banks cost about the same on either store, and a small deployment can be cheaper on Postgres. If your banks stay under roughly 50,000 memories each, Postgres is a good fit.
First reads are slower: Object storage makes warm reads cheap and first reads slow. Servers preload what recall needs whenever they open a bank, banks stay routed to the same server, and one early recall warms a bank you know is about to get busy.
Moving from Postgres: A migration copies an organization while it stays live, keeps every memory ID, and verifies every bank record by record before switching. The original copy is kept, so it can be moved back.

Availability

Part of Hindsight Enterprise, wherever it runs

Memlake plugs into Hindsight’s storage layer. Your agents, SDKs and API calls don’t change.

Hindsight Cloud: Vectorize runs Memlake for you. Nothing to set up.
Private cloud: A dedicated deployment in our cloud account, with buckets dedicated to your organization.
Bring your own cloud: Your AWS, Google Cloud or Azure account, on S3, GCS or Azure Blob Storage, with your encryption and access policies.
On-premises: Your data center, on S3-compatible storage certified for the conditional writes Memlake depends on.

Compare Enterprise deployment options

Documentation

Go deeper

The launch post: Why we built Memlake, how it works, and the numbers, in one read.
Overview: How Memlake stores your data, when to use it, enabling it, backups and security.
Architecture: The bucket layout, the write, index and query paths, and how the parts coordinate.
Guarantees: The guarantees Memlake gives, from durable writes to isolation, and how we check them.
Performance and cost: Latency, throughput and cost numbers, and how they were measured.
Tradeoffs: What Memlake trades away, and when Postgres is the better fit.
Concepts: A glossary of the terms used across the Memlake docs.