> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anonalabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Prove the memory is earning its place

> Run the same questions with memory on and off, and keep the number instead of the impression.

[← All use cases](/use-cases/overview)

<div className="uc-examples"><a href="https://github.com/anonalabs/Anona-Memory-SDK/blob/main/examples/drop_in_proxy.py">drop\_in\_proxy.py</a><a href="https://github.com/anonalabs/Anona-Memory-SDK/blob/main/examples/memory_basics.py">memory\_basics.py</a></div>

## The scenario

Six weeks in, somebody asks whether the memory layer is actually helping. The
honest answer is usually "it feels like it" — which is not an answer, and it is
the same answer you would give if it had silently broken a month ago.

The drop-in proxy makes this measurable in a way most stacks cannot, because
memory is one header rather than an architecture.

## Step 1 — Turn memory off for one call

```python theme={null}
off = llm.chat.completions.create(
    model="balanced",
    messages=[{"role": "user", "content": question}],
    extra_headers={"X-Anona-Memory": "false"},
)
```

`X-Anona-Memory: false` skips recall and injection for that call only.
Everything else — model, prompt, parameters — is identical, which is what makes
it a comparison rather than two anecdotes.

## Step 2 — Run the same question with it on

```python theme={null}
on = llm.chat.completions.create(
    model="balanced",
    messages=[{"role": "user", "content": question}],
)
print(on.memories_injected, "memories injected")
```

## Step 3 — Score a real set, not one question

```python theme={null}
for question, expected in eval_set:
    a = ask(question, memory=False)
    b = ask(question, memory=True)
    record_result(question, a, b, expected)
```

Twenty to fifty questions drawn from real traffic, where the answer depends on
something said in an earlier session. Questions answerable from the current turn
alone will score identically and tell you nothing — that is the control.

## Step 4 — Keep running it

Put the eval set in CI. A recall regression is otherwise invisible: nothing
errors, answers just quietly get worse.

## Evals

1. Build the set from real conversations, including five questions memory
   *cannot* help with.
2. Expect no difference on those five. A difference there means something else
   is varying, and your measurement is not measuring what you think.
3. Report per-question, not just an average — memory usually helps a lot on a
   few and not at all on most, and the average hides both.
4. Re-run after any change to scoping, extraction settings or recall tuning.

## Guardrails

<Warning>
  **Do not publish a benchmark number without the harness that produced it.**
  This category has already seen a headline score fall by twenty points under
  independent measurement. An internal number is for deciding; a public number
  needs a script someone else can run.
</Warning>

* **Both arms cost credits.** An A/B over fifty questions is a hundred calls,
  and the memory-off arm bills like any other.
* **`memory: false` still records the turn** unless you also set
  `X-Anona-Auto-Record: false` — so a naive A/B slowly teaches the space the
  answers to its own eval set. Turn both off for the control arm.
* **Fresh spaces score badly and that is correct.** Memory helps in proportion
  to what it has seen; measure on a space with real history.

## Built from

Both are complete, runnable scripts:
[`drop_in_proxy.py`](https://github.com/anonalabs/Anona-Memory-SDK/blob/main/examples/drop_in_proxy.py),
[`memory_basics.py`](https://github.com/anonalabs/Anona-Memory-SDK/blob/main/examples/memory_basics.py).