A chat assistant memory system stores useful facts and preferences about a user and recalls them in later conversations, so the user doesn't have to keep repeating the same information.
Assistants with memory typically include a settings page where users can review what has been saved, make corrections, and delete anything they don't want kept.
Popular Examples: ChatGPT, Gemini, Claude
Without memory, a chat assistant starts each new conversation without the personal context shared in earlier ones. A user who mentioned last week that they're vegetarian, use Java at work, and prefer short answers may have to repeat those details when asking for a recipe or a code review.
A basic implementation stores a few notes per user and adds them to the prompt. A reliable system also needs to decide what's worth remembering, recognize when a fact has changed, keep personal details out of unrelated conversations, and ensure that deleted memories stop being used.
In this chapter, we'll design a memory system for a consumer chat assistant that stores billions of memories. We'll start with the requirements and high-level architecture, then examine what qualifies as a memory, how to handle conflicting and outdated facts, how deletion works, and how to measure whether memory improves the experience.
Let's begin by clarifying the requirements.