Hindsight Models

Models trained for Hindsight

A decision model, a reranker and an embedding model, each trained for one job inside Hindsight. Built for that job, they stay lightweight, so memory gets faster and spends less on LLMs. Run the open weights yourself, or point your self-hosted Hindsight at Hindsight Inference.

Try Hindsight Inference Open weights on Hugging Face

How the models work →

cheaper where the decision model takes parths-decide-1-flash: 2.5×
less time in model calls, with better answershs-decide-1-flash: 2×
faster on CPU than gte-reranker-modernbert-base, and more accuratehs-rerank-1: 6×
mean over 12 benchmarks, highest of the small open embeddershs-embed-1: 56.8

Cost and speed: Hindsight’s system evals with and without hs-decide-1-flash, on seven LLMs. The largest saving of the seven, on consolidation, reflect and knowledge pages; speed is time spent in model calls. Model comparisons: public benchmarks, on the model leaderboards.

The models

Three models, each trained for one job in Hindsight

Each one handles a step of memory that would otherwise run on a general-purpose model. They’re lightweight and fast because they’re built for that step, and competitive with widely used open models on public benchmarks we didn’t make.

Decision model

hs-decide-1-flash

Answers the small, typed questions you would otherwise send to an LLM: pick one, how likely, where on a scale. A probability for every option, no text to parse.

In Hindsight

Consolidation, reflect and knowledge pages, and recall if you turn it on

Takes over the small judgments that would otherwise go to an LLM. The biggest saving: up to 2.5× cheaper and 2× less time in model calls, with better answers from six of the seven LLMs we tested.

Parameters: 0.8B
Context: 64k tokens
Languages: Multilingual

75.6

Mean over 8 decision benchmarks

ahead of StartLux-Decision 0.8B Instant (74.5) and d3-lite (73.4)

Hugging Face All results

Reranker

hs-rerank-1

A multilingual cross-encoder that puts the memory that answers the question first. The default reranker in open-source Hindsight, and fast on a CPU.

In Hindsight

Every recall, and every reflect and page refresh that searches through it

Puts the memory that answers the question first, so the LLM reads less and gets the right context. It reranks 50 memories in about 1.5 s on two CPU threads, 6× faster than gte-reranker-modernbert-base.

Parameters: 114M
Context: 8k tokens
Languages: Multilingual

56.0

Mean over 7 reranking benchmarks

ahead of gte-reranker-modernbert-base (51.3) and bge-reranker-base (50.4)

Hugging Face All results

Embeddings

hs-embed-1

A multilingual embedding model that turns text into vectors for search by meaning. Light enough to run on a CPU.

In Hindsight

Every memory retained and every recall query

Finds memories by meaning, across languages. Trained on long conversations and logs, and light enough to run on a CPU, so search by meaning stays fast and cheap at any volume.

Parameters: 112M
Dimensions: 384, 256, 128 or 64
Languages: Multilingual

56.8

Mean over 12 retrieval benchmarks

ahead of granite-embedding-97m-multilingual-r2 (54.6) and multilingual-e5-small (50.3)

Hugging Face All results

hs-embed-1: the most accurate, and among the fastest

Mean over 12 retrieval benchmarks against median query latency on two CPU threads, for seven small open embedding models.

hs-rerank-1: the most accurate, at a sixth of the time

Mean over 7 reranking benchmarks against the median time to rerank 50 memories on two CPU threads, for six open rerankers.

Every number on the model leaderboards comes from runs we made ourselves, with the same data and settings for every model.

The biggest saving: hs-decide-1-flash

The LLM writes. The decision model judges.

Most of what a memory system does isn’t writing text. It’s small calls: is this fact new, which of these memories matter, is this evidence enough, does this page need a rewrite. Sending each one to a large LLM is slow and costs tokens. hs-decide-1-flash answers them in milliseconds, for a fraction of the price. Of the three models it saves the most, because every decision is an LLM call Hindsight doesn’t make.

Seven LLMs, each alone and with hs-decide-1-flash

Hindsight's system evals on consolidation, reflect and knowledge pages, the operations the decision model takes part in. Quality is the mean share of cases passed; cost is the tokens used at list prices, decision-model tokens included. Each arrow runs from an LLM alone to the same LLM with the decision model: six of seven point up, and five point left.

Recall

Which memories matter?

Without a decision model: Recall fills its token budget whether the memories help or not, and your agent’s own LLM reads all of it.
With hs-decide-1-flash: hs-embed-1 finds candidates by meaning and hs-rerank-1 orders them. Turn on the decision model and it drops memories that don’t help and repeats of better ones, so your agent reads fewer tokens.

Optional: on per request or per bank

Reflect

Is this evidence enough?

Without a decision model: An LLM agent runs every round: it picks a tool, writes a query, reads every result and decides whether to search again.
With hs-decide-1-flash: The decision model trims the evidence and judges when it answers the question. The LLM only writes the query and the answer.

4.4 → 2.5 LLM turns per question; up to 4.6× cheaper

Consolidation

Is this fact new?

Without a decision model: Every new fact goes to the LLM, along with every observation it might touch.
With hs-decide-1-flash: Facts that restate what the bank believes, or have nothing to do with it, are settled without the LLM. Only facts that change or add to it reach the LLM.

More accurate observations with all seven LLMs tested; up to 2.3× less time in model calls

Knowledge pages

Could this change the page?

Without a decision model: Every update runs a full reflect over the new memories and rewrites the page.
With hs-decide-1-flash: Only the memories that could change the page are used, and when none can, there is no LLM call at all.

About 8.5 → 4 LLM calls per case; up to 3.6× cheaper

Recall also runs on hs-embed-1 and hs-rerank-1, and every reflect and knowledge-page refresh searches through recall, so all of them gain from the three models together. If the decision model is unavailable, Hindsight falls back to the LLM.

Trained to purpose

Lightweight, because each does one job

Every model is trained for its step in Hindsight, not for everything a general model might be asked. That is what lets all three be light enough to answer in milliseconds and cheap enough to call on every step.

Built for a narrow job: A general-purpose model carries the weight of everything it might be asked. Ours are trained on what Hindsight actually does, so the embedder and reranker are about 110M parameters and the decision model is 0.8B.
Fast enough for every step: The embedder and reranker are light enough to run on a CPU, and a decision comes back in 167 ms at the median. Fast enough to use on every recall, reflect and page update, not just the ones that can afford it.
Each step on the right model: Search by meaning goes to the embedder, ordering to the reranker and judgments to the decision model. Your LLM is kept for what needs it: extracting memories and writing answers and pages.
Ours to keep improving: Because we train them, the models improve as memory gets harder: longer histories, more agents sharing a bank, more sources that disagree.

Hindsight Inference

The same models, for your self‑hosted Hindsight

Hindsight Inference is our hosted service for the models. Point your self-hosted Hindsight at it with a one login and one environment variable and get the decision model without running any GPUs, at inference.hindsight.vectorize.io.

Part of Hindsight Cloud: same login, organization, teams and billing
One login and one environment variable in Hindsight, no GPU
Also works from any application: OpenAI, Cohere and TypeSafe compatible
Pay per million input tokens. Output is free
New accounts start with $5 of free credit

Model	Compatible with	Per 1M tokens
hs-decide-1-flashPOST /v1/systemone	TypeSafe System One	$0.02
hs-rerank-1POST /v1/rerank	Cohere Rerank	$0.015
hs-embed-1POST /v1/embeddings	OpenAI Embeddings	$0.005

Try Hindsight Inference Inference docs Pricing

Ask a decision

curl https://inference.hindsight.vectorize.io/v1/systemone \
  -H "Authorization: Bearer $HINDSIGHT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hs-decide-1-flash",
    "state": "Question: When is the offsite?",
    "questions": {
      "answer": {
        "type": "choice",
        "instructions": "Which note answers the question?",
        "criteria": {
          "c0": "The offsite moved to April 17.",
          "c1": "Priya prefers window seats."
        }
      }
    }
  }'

A typed answer back

{
  "answers": {
    "answer": {
      "choice": "c0",
      "probabilities": { "c0": 0.95, "c1": 0.05 }
    }
  }
}

Models trained for Hindsight

Three models, each trained for one job in Hindsight

hs-decide-1-flash

hs-rerank-1

hs-embed-1

The LLM writes. The decision model judges.

Recall

Reflect

Consolidation

Knowledge pages

Lightweight, because each does one job

The same models, for your self‑hosted Hindsight

Hosted, self-hosted or inside Hindsight

Make memory cheaper and faster