Plan: “Agent Memory, Made Clear” — a product-grade mini-course
Goal: after reading this series, you can launch a product around agentic/LLM memory — you understand every memory type, every storage and retrieval choice, how the leading systems work internally, how to optimize cost/latency, how to evaluate, and how to defend against attacks.
Style: same as the scaling/ series — every sentence clear, nothing assumed, each part ~5-8 min, built strictly on the parts before it.
Research inputs (done before writing)
- Taxonomy: the field has converged on the cognitive-science taxonomy formalized by the CoALA framework: working (context window), episodic, semantic, procedural memory.
- Mem0: two-phase pipeline — LLM extracts candidate facts, then a consolidation step chooses ADD / UPDATE / DELETE / NOOP against similar existing memories; graph variant extracts entity-relationship triplets; reports ~26% accuracy boost, ~91% lower p95 latency, ~90% token savings vs full-context.
- Letta / MemGPT: OS analogy — main context (in-context) vs external context; editable in-context “memory blocks” (persona, human) with size limits; the agent edits its own memory via tools; “sleep-time compute” = a background agent that reflects, consolidates, and rewrites memory between sessions.
- Zep / Graphiti: temporal knowledge graph; every edge carries validity intervals (t_valid, t_invalid) plus ingestion timestamps (bi-temporal); contradictions invalidate old facts instead of deleting them, so history is preserved and point-in-time queries work.
- Generative Agents (Stanford): memory stream scored at retrieval by recency (exponential decay) + importance (LLM-rated) + relevance (embedding similarity); periodic “reflection” produces higher-level memories.
- Voyager / skill libraries, Reflexion: procedural memory as stored, reusable skills and self-critique lessons.
- Context engineering (Anthropic): context rot; compaction (summarize + restart), structured note-taking to files, sub-agents that burn tokens privately and return distilled summaries.
- Products: ChatGPT memory = implicit auto-saved profile (~40-80 facts typical); Claude memory = conversations synthesized (~daily) into a visible, editable profile, citations when used, deletable; Claude Code = CLAUDE.md + auto-memory directory + auto-compaction.
- Optimization: production stacks bias to precision — multi-signal retrieval (vector + keyword + graph + recency + metadata) at ~7k tokens/query vs 25k-100k for full-context, >91% recall; decay/eviction policies need access-count instrumentation.
- Evaluation: LoCoMo (1,540 Qs, ~300-turn multi-session conversations), LongMemEval (500 Qs across 6 abilities incl. knowledge update + temporal reasoning), BEAM (1M-10M token scale).
- Security: memory poisoning — MINJA (query-only injection, >95% success), AgentPoison (RAG-store poisoning, ~62%), MemoryGraft (10 seeds → 48% poisoned retrieval); defenses: write-time filtering, provenance binding, retrieval-time checks, isolation of untrusted content.
The blogs (13 parts + README + map)
| # | File | Teaches |
|---|---|---|
| 1 | 01-why-agents-need-memory.md | LLMs are stateless (the goldfish problem); weights vs context vs external store; what breaks without memory |
| 2 | 02-working-memory-the-context-window.md | The context window as working memory; why “just use a bigger context” fails (cost, context rot) |
| 3 | 03-the-four-types-of-memory.md | The full taxonomy: working / episodic / semantic / procedural, with one running example |
| 4 | 04-episodic-memory.md | Storing experiences; raw logs vs summaries; reflection; learning from past successes/failures |
| 5 | 05-semantic-memory.md | Facts and profiles; extraction; profile-vs-collection; knowledge triplets; updating and contradictions |
| 6 | 06-procedural-memory.md | Skills, instructions, self-edited prompts; CLAUDE.md; skill libraries; lessons-learned loops |
| 7 | 07-how-memories-are-stored.md | The four substrates: files, vector DB (embeddings explained from zero), knowledge graph, SQL; hybrid |
| 8 | 08-how-memories-are-retrieved.md | The read path: when to retrieve, query building, scoring (similarity+recency+importance), reranking, injection |
| 9 | 09-writing-consolidating-forgetting.md | The write path: what to save, hot-path vs background, ADD/UPDATE/DELETE/NOOP, dedup, decay, eviction, sleep-time compute |
| 10 | 10-how-modern-systems-do-it.md | Case studies: MemGPT/Letta, Mem0, Zep/Graphiti, LangGraph/LangMem, ChatGPT & Claude memory, Claude Code |
| 11 | 11-context-engineering-and-optimization.md | Compaction, note-taking, sub-agents, token budgets, prompt caching, the cost math of memory |
| 12 | 12-evaluating-and-securing-memory.md | Benchmarks (LoCoMo, LongMemEval, BEAM), metrics, failure modes, memory poisoning + defenses, privacy |
| 13 | 13-build-your-own-memory-product.md | The blueprint: minimal viable memory → advanced roadmap; architecture; decision checklists; pitfalls |
Plus:
- README.md — series index table.
- memory-map.svg — one-page hand-drawn-notebook-style map (like the trading-strategy note): title box, idea box, numbered sections (types, the read/write loop diagram, storage table, how modern systems do it, optimization pros/failure modes, when-to-use), yellow summary box.
Product-coverage checklist (what “launchable” means)
- [x] All memory types + when each is needed (parts 3-6)
- [x] Full write path and read path design (parts 8-9)
- [x] Storage technology choice (part 7)
- [x] Optimization: tokens, latency, cost, forgetting (parts 9, 11)
- [x] Competitive landscape internals (part 10)
- [x] Evaluation story for the product (part 12)
- [x] Security & privacy story (part 12)
- [x] Architecture blueprint + roadmap (part 13)