LLMs have no memory between requests. Every call to the model starts from a blank slate — the model only knows what you include in the current prompt. This is fine for single-turn Q&A, but it fails immediately for real agent tasks.
In this lesson, you will learn what goes wrong without conversation memory and what short-term memory needs to provide.
What fails without conversation memory
Consider an agent helping a customer service representative:
-
The representative asks: "Review Jessica Norris’s account for a credit limit increase."
-
The agent retrieves the account details.
-
The representative follows up: "What was the last transaction on that account?"
Without memory, step 3 fails. "That account" refers to context from step 1, but the model has no access to it. The agent must ask the representative to repeat themselves, or the application must manually re-inject the full conversation history on every call.
Manual history injection works — but it has no structure. The result is a wall of text, not a traversable graph. You cannot ask "which messages in this session mentioned Jessica Norris?" or "find the message that triggered this reasoning trace." You are re-reading a log.
What short-term memory provides
Short-term memory solves this with three capabilities:
-
Context injection — the most recent messages are retrieved and injected into each new prompt automatically. The agent always has the current conversation context.
-
Semantic search — past messages are embedded and indexed. You can find messages that discuss the same topic even if they use different words.
-
Graph connectivity — messages are linked to the entities mentioned in them and to the reasoning traces they triggered. You can traverse from a message to what the agent knew and what it decided.
Understanding the session boundary
Short-term memory is scoped to a session. When the session ends, the conversation history is no longer the agent’s active working memory — but the entities extracted from it persist in long-term memory. The session ends; the knowledge remains.
Summary
In this lesson, you learned why short-term memory is necessary:
-
Stateless LLMs — without memory management, every prompt is a fresh start; multi-turn agent tasks fail
-
More than a log — short-term memory provides context injection, semantic search, and graph connectivity
-
Session-scoped — short-term memory is ephemeral, but entity knowledge extracted from it persists to long-term memory
In the next lesson, you will see the graph schema that implements short-term memory.