PHASE 08 · MEMORY
Goal: Move from stateless chat to persistent, personalized interactions.
An LLM call is stateless — the model forgets everything between requests. Memory is the system you build around the model to make it feel persistent.
There are two tiers. Short-term memory is the conversation history within a single session: you store messages in a list, pass the last N turns into each new prompt. Long-term memory is facts about the user that survive across sessions: their name, preferences, past projects, recurring topics. Stored in SQLite or a vector DB keyed by user_id, loaded into the system prompt at session start.
The third concept is identity: the user profile + persistent system prompt that makes "this agent" feel like this agent. And as conversations grow long, you'll need compaction: replace old turns with a summary so you don't blow the context window.
session_id. Pass the last N turns into each new prompt.user_id and timestamp.memory.py — The agent remembers across sessions, summarizes long chats, and personalizes replies.
Concepts unlocked: memory tiers · compaction · user modeling · identity
Back in Phase 1 you learned the most jarring fact in this course: the model remembers nothing between calls. Every single API call starts from a blank slate, and the "conversation" is an illusion your code creates by re-sending the whole transcript every time. So how does a chatbot greet you by name, or pick up a project you mentioned last week? It doesn't. You build the memory around it, and feed the right bits back in each time.
Here's the analogy that carries this whole phase. Imagine a brilliant assistant with total amnesia — sharp as anyone while you're talking, but the moment a conversation ends, it's gone. Their fix: a notebook. Before every reply, they re-read their notes. After every conversation, they jot down what mattered. Ask them next week and they answer instantly — not because they remember, but because they read it back to themselves a second ago. That notebook is what "memory" means for an agent: text it saves, then re-loads into the prompt.
One word to pin down before the example: a session is one continuous conversation — from the moment you open the chat to the moment you close it. Within a session, your code holds the growing message list. Between sessions, only what got written to the notebook survives.
What memory really is. The model is stateless — nothing carries over between calls. "Memory" is the notebook system you build around it: well-chosen text, saved after conversations and loaded back into the prompt. The skill of this phase is choosing what to write down and what to read back.
So the agent keeps a notebook. That notebook has two distinct parts — the page it's writing on right now, and the saved notes in the back. In agent terms, memory comes in two tiers:
Why both kinds? They answer different questions. A saved fact answers "who is this person?" — true today, true next month. A session summary answers "what were we doing?" — the story so far, one chapter per conversation. An agent that loads only facts knows Maya but forgets the project; one that loads only summaries knows the project but keeps asking how she'd like to be contacted.
"Loaded into the system prompt" is a bit abstract — here's what it literally means. This is the actual text sent to the model, assembled from both tiers, before Maya's new question is even added:
You are Zentara Logistics' support agent, talking to Maya.
Known facts (long-term):
- Prefers email over phone
- Manages the Atlanta warehouse
Last session summary (long-term):
- Set up her refund workflow: refunds under $50
auto-approve, larger ones go to a human.
She wants to add email notifications next.
Recent messages (short-term, this session):
- user: "can we pick up where we left off?"
- assistant: "Sure — ready to add those email
notifications to the refund workflow?"That's the whole trick, in full: just text, assembled from the notebook, pasted in front of the conversation. Nothing about the model changed — it's still the amnesiac from Phase 1. You just handed it the right reminders before it started predicting.
The whole trick behind "it remembers me". At the start of a new session, load the user's saved facts and a short summary of last time into the system prompt. The agent picks up right where it left off — without remembering a thing.
Short-term memory has a catch. Remember from Phase 1 that the whole transcript is re-sent on every call — and the model can only read so much at once. That limit is its context window: the maximum amount of text one call can hold. A busy support chat can run to hundreds of turns; eventually the transcript simply won't fit. And even before it stops fitting, it hurts — every extra turn is re-sent (and paid for) on every single call.
The fix is compaction. Think of the notebook again: when the page fills up, you don't copy every old line onto a fresh page — you write one condensed note at the top ("so far we've…") and keep going. Compaction does exactly that to a chat: summarize the oldest turns into a few sentences, replace them with that summary, and keep the recent turns word-for-word. You trade exact wording for room.
Watch it happen — messages fill the window, and the oldest ones squeeze into a single summary card:
Here's a real Zentara support session, six messages in and pushing the limit:
1 user: "A customer on order #4127 wants a refund —
the item arrived damaged."
2 assistant: "Got it. Order #4127 is $38, under the $50
auto-approve limit. Refund issued."
3 user: "She asked if we can send a replacement
instead next time."
4 assistant: "Noted — I've marked replacement-first as
her preference."
5 user: "New topic: what's our policy on late
international shipments?"
6 assistant: "Zentara refunds shipping fees on international
orders that arrive more than 10 days late."Compaction asks the model to summarize the oldest four messages, then swaps them out. This is what the list looks like afterwards:
★ [summary of msgs 1–4]:
"Issued a $38 auto-approved refund for damaged
order #4127. Customer prefers a replacement over
a refund in future cases."
5 user: "New topic: what's our policy on late
international shipments?"
6 assistant: "Zentara refunds shipping fees on international
orders that arrive more than 10 days late."Look at what survived and what didn't. The facts made it: the order number, the $38, the replacement preference — if message 7 asks "what did we refund on #4127?", the agent can still answer. The exact wording is gone. And the two most recent turns stayed untouched, word-for-word, because the current thread is where precision matters most.
Compaction is lossy. Whatever the summary drops, the agent has truly forgotten — there's no original to go back to inside the chat. That's why compaction targets the oldest turns and keeps recent ones verbatim: the further back a detail is, the less likely you need its exact wording.
And here's the neat part: long-term memory is built with the very same move. At the end of a session, you ask the model to write the session summary (compaction's trick, applied to the whole chat) and to pull out durable facts ("prefers email over phone") to save. One habit, two payoffs: summarize to remember; extract to personalize.
Compaction in one line. Compaction = replace the oldest turns of a long chat with a short model-written summary, so the conversation keeps fitting in the context window. The gist stays; the space comes back.
Time to give your agent its notebook: short-term within a session, long-term across sessions, and compaction for chats that run long.
Read what it builds — the notebook routine in code: memory loaded into the prompt at the start of a session, written back at the end:
# memory.py — make a stateless model feel persistent
def build_prompt(user_id, session_id, question):
facts = load_facts(user_id) # long-term: saved facts
summary = last_summary(user_id) # long-term: last session's recap
recent = last_turns(session_id, n=8) # short-term: this session
system = f"You are talking to {facts}. Last time: {summary}"
if too_long(recent):
recent = compact(recent) # summarize oldest turns to fit
return system, recent + [{"role": "user", "content": question}]
def end_session(user_id, session_id):
facts = extract_facts(history(session_id)) # ask the model what to remember
save_facts(user_id, facts)
save_summary(user_id, summarize(history(session_id)))Deliverable — memory.py. The agent remembers across sessions, compacts long chats so they fit, and personalizes its replies — built entirely from saving and re-loading text around a model that still remembers nothing.