DW Recall Eval Digest

_Run 2026-09-24T09:55:25+00:00 • http://localhost:8080/api/v1/memories • queries v1_

Verdict: 🟡 NEUTRAL

_Baseline: 2026-08-08T06-15-40+00-00.json_

No improvements vs baseline AND no regressions — neutral run.

Latency p95: baseline 297.1ms → current 209.5ms (budget 445.7ms)

Per-query status vs baseline

Query	Status	Baseline newest	Current newest	Δ
`heartbeat-wide`	STABLE	6.6min	2.8min	+3.8min
`heartbeat-tight`	STABLE	6.6min	2.8min	+3.8min
`heartbeat-mid`	STABLE	6.6min	2.8min	+3.8min
`global-no-filter`	UNMEASURABLE	—	—	—
`sort-param-respected`	STABLE	6.6min	2.8min	+3.8min

5 queries • 0 with flags

Aggregate latency (all samples) p50=193.5ms p95=210.2ms max=209.5ms

`heartbeat-wide`

_The canonical heartbeat-freshness query from CLAUDE.md. limit=200, no sort. Must return cl-pal + codex-pal with fresh ts._

params: {"user_id": "mop-heartbeat", "limit": 200}
latency: p50 204.1ms, max 206.0ms (3 samples)
result_count: 200, http=200
agents_seen: cl-pal, codex-pal
freshness_min_per_agent: cl-pal=2.7min, codex-pal=3.5min
first_id_vs_prior: churned (was 1795ac23-c181-4427-88fe-d1165cbb28d9, now 1d472778-f582-480f-bde6-e15d52be04a0)
✓ no flags

`heartbeat-tight`

_Catches the 'limit=5 silently returns arbitrary subset' bug. Same query as wide but limit=5 — record what set comes back so we can see set instability across runs._

params: {"user_id": "mop-heartbeat", "limit": 5}
latency: p50 190.9ms, max 209.5ms (3 samples)
result_count: 5, http=200
agents_seen: cl-pal, codex-pal
freshness_min_per_agent: cl-pal=2.7min, codex-pal=3.5min
first_id_vs_prior: churned (was 1795ac23-c181-4427-88fe-d1165cbb28d9, now 1d472778-f582-480f-bde6-e15d52be04a0)
✓ no flags

`heartbeat-mid`

_Middle-bucket size to triangulate where set instability kicks in._

params: {"user_id": "mop-heartbeat", "limit": 50}
latency: p50 196.4ms, max 200.1ms (3 samples)
result_count: 50, http=200
agents_seen: cl-pal, codex-pal
freshness_min_per_agent: cl-pal=2.8min, codex-pal=3.6min
first_id_vs_prior: churned (was 1795ac23-c181-4427-88fe-d1165cbb28d9, now 1d472778-f582-480f-bde6-e15d52be04a0)
✓ no flags

`global-no-filter`

_Unfiltered fetch to track corpus size + global latency. Tracks total memory store growth._

params: {"limit": 200}
latency: p50 193.5ms, max 203.5ms (3 samples)
result_count: 200, http=200
first_id_vs_prior: churned (was cea6a6c9-0fad-4a0e-a162-0f213655ad76, now 2f7f82cb-974b-4909-b788-68ea30a46e3d)
✓ no flags

`sort-param-respected`

_Per CLAUDE.md, the endpoint silently ignores sort/order/order_by params. This query passes sort=ts:desc — if behavior ever changes (endpoint starts respecting sort), the first-returned-id should differ vs heartbeat-wide. v1 just records the fact; v2 (with Steward) should pass._

params: {"user_id": "mop-heartbeat", "limit": 5, "sort": "ts:desc"}
latency: p50 190.3ms, max 205.4ms (3 samples)
result_count: 5, http=200
agents_seen: cl-pal, codex-pal
freshness_min_per_agent: cl-pal=2.8min, codex-pal=3.6min
first_id_vs_prior: churned (was 1795ac23-c181-4427-88fe-d1165cbb28d9, now 1d472778-f582-480f-bde6-e15d52be04a0)
✓ no flags

How to read this

Flags are expectation-deltas, not hard failures (timer never pages).
first_id_vs_prior=churned on small-limit queries is the documented silent-arbitrary-subset behavior; it should stabilize once Steward re-rank lands.
Aggregate p95 is the baseline budget — Steward must not blow this out.
Historical snapshots live in recall_eval/baselines/*.json for offline diffing.