AI/ML●●Solid
Benchmarking the Keep memory system with LoCoMo
Local-only memory benchmarking—rigorous, but LoCoMo is a standard test everyone runs.
Niche GemShip It
inguz
103mo ago
Back to browse
GitHub Repository
WMB-100K — The first 100,000-turn benchmark for AI memory systems
12 starsRust
WMB-100K – Open benchmark for AI memory systems at 100K turns
by wontopos·Apr 1, 2026·2 points·0 comments
AI Analysis
●●●BangerBig BrainNiche Gem
100K-turn benchmark tests situational memory retrieval where others stop at 600.
Strengths
- •Simulates 4.3M token context window, dwarfing prior benchmarks at 50K.
- •Situational questions test retrieval relevance, not just keyword fact lookup.
- •Includes 400 false memory tests to check hallucination defense specifically.
Weaknesses
- •Niche audience limited to AI memory system researchers and builders.
- •Scoring requires LLM judges, adding cost and latency to evaluation.
Category
Target Audience
AI researchers and engineers building long-term memory systems
Similar To
LOCOMO · LongMemEval · Needle In A Haystack
Similar Projects
AI/ML●●Solid
Polyvia – Multimodal document retrieval over 100K+ files
Sub-200ms retrieval over 100K files when most RAG systems choke at 100.
SlickSolve My Problem
mgierlach-polyv
301d ago
Productivity●●Solid
Manex Hub – private AI research memory, on-device
On-device research memory that saves conversations back into the graph for compounding knowledge.
Niche GemCozy
krcnow
202mo ago
AI/ML●●Solid
Pallium, a local-first memory sidecar for agent conversations
Structured memory sidecar beats raw vector search for agent continuity.
Big BrainShip It
margolis20
102mo ago
AI/ML●Mid
One memory layer across every MCP-compatible AI tool
Another memory layer for MCP when Mem0 and Zep already solve this.
Solve My Problem
gavinb-code
201mo ago
AI/ML●Mid
ChatJimmy Ultra (100k+ tokens/sec)
In-browser LLM inference, but unclear if 100k tok/sec is real or marketing.
Ship ItBig Brain
jstanley
103mo ago