Given a short chat history and a follow-up question that depends on it, use the model to rewrite the follow-up into a self-contained search query that makes sense without the conversation.

In a multi-turn chat, users lean on context: they say "what about its pricing?" or "and the free tier?" instead of repeating the subject. A retriever embeds the query on its own, with no memory of earlier turns, so a pronoun-laden follow-up retrieves the wrong chunks. Rewriting the follow-up into a standalone query before retrieval is the fix, and it is cheap enough to run on every turn with a small model.

Requirements

Expected Output

The pronoun is resolved, and the rewritten query stands on its own.

Solution

The history is flattened into the prompt so the model can see what "it" refers to, and the system prompt constrains the output to just the query, with no preamble. This rewrite step sits between the user and the retriever: every turn the raw follow-up gets turned into a query that retrieves correctly on its own, which is what makes retrieval work in a real conversation rather than only on isolated questions.

Rewriting a follow-up is only worthwhile if it improves retrieval. Feed both the raw follow-up and the rewritten standalone query into the retriever and compare what each pulls back. The pronoun-laden original often retrieves the wrong chunk, while the rewrite finds the right one.

Requirements

Expected Output

The raw "how does it handle persistence?" retrieves poorly without context, while the rewritten "Redis persistence" query finds the persistence chunk.

Solution

The raw follow-up has no subject, so its embedding drifts toward whatever database text is nearby, sometimes the wrong one. Resolving "it" to "Redis" before embedding aligns the query with the right chunk. Seeing both retrievals side by side makes the value concrete: the rewrite step is not cosmetic, it directly changes which evidence reaches the model, which is why conversational RAG puts it before retrieval on every turn.