The Write Path: Saving, Consolidating, and Forgetting

Agent Memory, Made Clear · Part 9 · 7 min read

Retrieval gets the glory, but the write path decides what exists to be retrieved. A memory store is a garden: what you plant (selection), how you plant it (consolidation), and what you prune (forgetting) determine whether, two years in, you have a garden or a landfill. Three disciplines, in order.

Discipline 1: Selection - what deserves to persist?

The naive policy - save everything - fails on every axis at once: cost (embedding and storing oceans of chit-chat), retrieval quality (needles now live in a bigger haystack, and Blog 8’s scoring must fight through trivia), and trust (users are rightly uneasy when every throwaway remark is permanent). The opposite policy - save only what the user explicitly says “remember this” about - misses nearly everything valuable, because people don’t narrate their own facts.

So real systems run LLM-judged selection (Blog 5’s extraction): after an exchange or a task, a model call asks “what here is worth keeping?” against explicit criteria. The criteria are the product decision; a good starter set:

Durability test: will this plausibly matter in a month? (“Maya migrated to FastAPI” - yes; “Maya said thanks” - no.)
Category allowlist: identity, preferences, projects, constraints, decisions, outcomes+lessons - and, per Blog 6, corrections (“stop doing X”) which are procedural gold.
Sensitivity rules: some categories (health, credentials, anything the user asked to keep off the record) should require explicit consent or never persist - a product-trust decision as much as a technical one.

Also part of selection: choosing the memory type (Blogs 3-6). “Broke staging on June 3” → episodic. “Team deploys Tuesdays” → semantic. “Always check config flags before deploy” → procedural. Same event, different shelves - and different shelf-lives.

Discipline 2: Timing - when does writing happen?

Three architectural options, with a real trade-off:

Hot path (write during the conversation). Extract-and-store as the exchange happens, or the agent itself calls a save_memory tool mid-task. Freshest possible memory - available to the next message. Cost: latency and tokens on the user’s clock, and split attention if the agent must both converse and curate.

Background (write after, asynchronously). The session ends; a background job reads the transcript, extracts, consolidates. The user never waits. This is where the heavyweight steps naturally live - reflection over episodes, profile synthesis (Claude’s memory synthesizes conversations into your profile on roughly a daily rhythm - a background write path at product scale).

Sleep-time compute (the background path, upgraded). The idea - pioneered by Letta - of a separate agent that works on memory while the main agent is idle: re-reading recent experience, forming higher-level “learned context,” resolving contradictions, reorganizing and rewriting memory blocks, pre-computing what tomorrow’s session will likely need. The metaphor is deliberate: it is what sleep does for human memory - the day’s raw experiences get replayed, consolidated, and filed. Agents get “sleep” for the same reason we do.

The standard production blend: cheap immediate capture (at minimum, don’t lose the raw transcript; optionally hot-path writes for corrections and explicit “remember this”) plus rich background consolidation on a schedule.

Discipline 3: Consolidation - the pipeline in full

Assembling the pieces from Blogs 5 and 8, here is the canonical write pipeline (this is essentially Mem0’s published two-phase design, and the shape most systems share):

new exchange
   │
   ▼
1. EXTRACT      LLM pulls candidate memories from the exchange
   │            (guided by selection criteria + recent context)
   ▼
2. RECALL       for each candidate: fetch top-K similar
   │            existing memories from the store
   ▼
3. RECONCILE    LLM compares candidate vs neighbors and picks:
   │            ADD (new) / UPDATE (merge-refine) /
   │            DELETE-or-INVALIDATE (contradiction) / NOOP (known)
   ▼
4. COMMIT       apply the operation; stamp metadata
                (time, source, confidence, validity interval)

Step 3 is the soul: it is what turns logging into maintaining beliefs. Skip it and the store fills with near-duplicates and dead facts - and retrieval (which can only see resemblance, remember) will happily serve the corpses. And per Blog 5’s timeline idea: prefer invalidate over delete when history matters - close the old fact’s validity interval so “what did we believe in June?” keeps an answer.

Background consolidation adds the longer-range moves on top: merge clusters of near-duplicate memories into one clean statement; summarize hierarchies (100 stale episodes → 1 monthly digest, raw kept in cold archive); promote across types (repeated episodes → semantic fact; repeated corrections → procedural rule - the refinery from Blog 4).

Discipline 4: Forgetting - the most neglected feature

Nobody wants to build forgetting; every long-lived system regrets not building it. Without it, stores grow without bound, stale beats fresh in retrieval fights, and cost creeps. Human forgetting isn’t a flaw - it is aggressive relevance filtering, and agents need the same. The toolkit, from gentle to final:

Decay (soft forgetting): Blog 8’s recency term already does it - unaccessed memories fade in ranking without being touched. Zero risk; deleting nothing.
Archival (medium): memories unaccessed for months move to cold storage - out of the active index (no longer pollute retrieval), still recoverable by deliberate deep search.
Eviction (hard): genuinely delete - typically only low-importance, long-unaccessed, superseded items, under a policy like “importance-weighted LRU” (least-recently-used, but importance buys immunity: the allergy fact never dies; the 2024 favorite-color remark can).
Summarize-then-drop: the graceful hybrid for episodes - distill the lesson, archive the raw, drop the middle.

Two operational rules make forgetting safe. Instrument first: eviction decisions need last_accessed and access_count on every memory - if your store doesn’t track them, add that before enabling any deletion. Verify after: a production trick - after each eviction pass, ask the agent its user’s most-accessed preferences and check the answers still hold; if they degraded, your policy is too aggressive. And regardless of policy: user-visible memories deleted by the user are deleted, everywhere, immediately. That one is not an optimization; it is the trust contract (and increasingly the legal one).

Quick recap

Selection: LLM-judged, criteria-driven (durability, categories, sensitivity), and type-aware - same event, different shelves.
Timing: hot path for freshness and explicit saves; background/sleep-time for the heavy consolidation; blend both.
Consolidation: extract → recall neighbors → reconcile (ADD/UPDATE/DELETE/NOOP) → commit with metadata; prefer invalidation over deletion; background jobs merge, summarize, and promote across types.
Forgetting is a feature: decay → archive → evict (importance-weighted LRU) → summarize-then-drop; instrument before evicting, verify after; user deletions are sacred.

We now have the full machine: types, storage, read path, write path. Next: how the leading real systems - MemGPT/Letta, Mem0, Zep, LangGraph, ChatGPT and Claude - actually assemble these pieces.