Episodic Memory: Learning From What Happened

Agent Memory, Made Clear · Part 4 · 6 min read

Episodic memory is the agent’s diary: specific experiences, tied to time. It sounds like the simplest type - “just save the transcripts!” - but doing it usefully involves three real design problems: what form to store, how to distill lessons, and how to bring the right episode back. Let’s take them in order.

Design problem 1: What form do we store an experience in?

An experience - say, one working session - can be saved at several altitudes:

Altitude 1: The raw transcript. Every message, every tool call, every output. Perfect fidelity, zero information loss. But a day’s agent session can be hundreds of thousands of tokens; recalling it into the context window later is impossible (it wouldn’t fit, it would cost a fortune, and Blog 2’s context rot would eat the value anyway). Raw logs are the archive, not the working memory format.

Altitude 2: The structured summary. After the session (or continuously), an LLM writes a compact record:

date: 2026-07-12
task: Add rate limiting to payments API
outcome: SUCCESS (2 attempts)
what happened: First attempt broke staging - RATE_CONFIG env var
  missing. Added flag to deploy checklist. Second attempt clean.
lesson: middleware changes need a config-flag check before deploy.
tags: payments, deploy, config

A few hundred tokens. This is the workhorse format of episodic memory: cheap to recall, easy to search, and it preserves the story - the sequence of attempt, failure, cause, fix.

Altitude 3: The one-line index entry. “July 12: rate limiting on payments - succeeded after config-flag incident.” Used for scanning many episodes at once.

Good systems keep more than one altitude: an index to scan, summaries to recall, raw logs to dig into when a summary isn’t enough (often exposed to the agent as a search tool over old transcripts). The pattern to remember: compress for recall, keep the raw for rare deep dives.

Design problem 2: Turning events into lessons - reflection

A pile of accurate summaries is a diary, not wisdom. The step that upgrades it is called reflection: periodically, an LLM re-reads recent episodes and asks “what patterns and lessons are in here?” - writing the answers back as new, higher-level memories.

This idea was made famous by Stanford’s Generative Agents work (the “AI town” of 25 simulated people, 2023): agents kept a stream of observations and periodically synthesized reflections like “Klaus is passionate about research” from many small events - then reflected on reflections, building a tree from concrete moments to real understanding of their world.

For a working agent, reflection over five deployment episodes might produce: “Three of the last five staging failures were missing environment variables → always diff env configs before deploying.” Notice what that is: episodic experience being distilled toward semantic memory (a fact about how deployments fail) and procedural memory (a new rule of behavior). Reflection is the refinery between the memory types - and it can run in the background, when the agent is idle. Blog 9 returns to this as “sleep-time compute.”

A close cousin from research: Reflexion (2023) showed that an agent which fails a task, writes itself a short verbal critique (“I failed because I assumed the file existed; next time check first”), and carries that note into the next attempt improves dramatically - self-generated lessons, stored and re-injected, function like a gradient update made of words.

Design problem 3: Bringing the right episode back

When Maya says “add caching to the orders API,” which episodes should surface? The mechanics of retrieval (similarity search, scoring) are Blog 8’s topic, but episodic memory adds its own special seasoning:

Similarity of task, not just words. The June 3rd rate-limiting episode is relevant to today’s caching task because both are “middleware change on a payment-adjacent service” - good episodic recall keys on task structure (tags like payments, deploy, middleware help enormously).
Recency matters more than for facts. Last week’s failure usually outranks last year’s; many systems explicitly decay old episodes.
Outcomes are gold. Episodes tagged SUCCESS are reusable recipes; episodes tagged FAILURE are warnings. Both flavors earn their tokens.

The quiet superpower: episodes as few-shot examples

Here is the most valuable and least obvious use of episodic memory. LLMs are famously good at few-shot learning - show a couple of examples of a task done well, and quality jumps.

Episodic memory makes the agent self-improving through examples: when a new task arrives, retrieve 1-2 successful past episodes of similar tasks and put them in the context as worked examples. “Here is how we did rate limiting in June (it worked); now do caching similarly.” The agent effectively trains itself on its own greatest hits - no fine-tuning, no weight updates, just a smart read path. Systems that do this report the same “experience-following” effect in both directions, which is worth engraving:

Agents imitate their retrieved past. Feed successes back and quality compounds; let junk episodes accumulate unfiltered and the agent faithfully repeats its own mistakes.

That is why write-time quality control (Blog 9) matters so much for this memory type: episodic memory is a curriculum the agent teaches itself from.

What episodic memory looks like in production

Concretely, a solid episodic subsystem for a real product:

During the session: append raw events to a log (cheap, always on).
At session end (or task end): LLM writes the structured summary - task, outcome, key events, lesson, tags - stored with a timestamp and embedding for search.
Periodically (background): reflect over recent summaries → distill patterns → write lessons, promote stable facts to semantic memory, promote behavior rules to procedural memory. Prune or archive episodes that reflection has already absorbed.
At task start (read path): retrieve top few similar episodes - prioritizing recency and successful outcomes - and inject as “relevant past experience.”

The failure modes to design against: storing only raw logs (unusable), storing without outcomes (can’t prefer successes), never reflecting (diary but no wisdom), and unbounded growth (Blog 9’s forgetting machinery is the cure).

Quick recap

Episodic memory = experiences with time and story attached; store at multiple altitudes - index, summary, raw - and recall the compact forms.
Reflection turns piles of episodes into lessons, and feeds the other memory types; Reflexion-style self-critiques make failures directly useful.
Retrieval for episodes weighs task-similarity, recency, and outcome.
The superpower: past successes recalled as few-shot examples make the agent self-improving - and the same mechanism faithfully amplifies junk, so curation is not optional.

Next: semantic memory - how a mountain of conversation gets distilled into clean, current facts, and what to do when facts change.