{"title":"Exercise: Conversational RAG","description":null,"content":"# Exercise 1: Rewrite a Follow-up into a Standalone Query\n\nGiven a short chat history and a follow-up question that depends on it, use the model to rewrite the follow-up into a self-contained search query that makes sense without the conversation.\n\nIn a multi-turn chat, users lean on context: they say \"what about its pricing?\" or \"and the free tier?\" instead of repeating the subject. A retriever embeds the query on its own, with no memory of earlier turns, so a pronoun-laden follow-up retrieves the wrong chunks. Rewriting the follow-up into a standalone query before retrieval is the fix, and it is cheap enough to run on every turn with a small model.\n\n### Requirements\n\n- Provide a chat history where the subject was established in an earlier turn.\n- Provide a follow-up question that uses a pronoun or omits the subject.\n- Ask the model to rewrite the follow-up into a standalone query that names the subject explicitly.\n- Print the rewritten query.\n\n\n**main.py**\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI() # Uses OPENAI_API_KEY from environment\n\nhistory = [\n {\"role\": \"user\", \"content\": \"Tell me about the Redis caching layer.\"},\n {\"role\": \"assistant\", \"content\": \"Redis is an in-memory store often used as a cache in front of a database.\"},\n]\n\nfollow_up = \"How does it handle persistence?\"\n\n# TODO: ask the model to rewrite follow_up into a standalone search query,\n# using the history to resolve what \"it\" refers to\n\n# TODO: print the rewritten query\n```\n\n\n### Expected Output\n\nThe pronoun is resolved, and the rewritten query stands on its own.\n\n\n```plaintext\nOriginal: How does it handle persistence?\nRewritten: How does Redis handle persistence?\n```\n\n\n### Solution\n\n\n**main.py**\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI()\n\nhistory = [\n {\"role\": \"user\", \"content\": \"Tell me about the Redis caching layer.\"},\n {\"role\": \"assistant\", \"content\": \"Redis is an in-memory store often used as a cache in front of a database.\"},\n]\n\nfollow_up = \"How does it handle persistence?\"\n\nconversation = \"\\n\".join(f\"{turn['role']}: {turn['content']}\" for turn in history)\n\nresponse = client.chat.completions.create(\n model=\"gpt-5.4-mini\", # A small, fast model is enough for rewriting\n messages=[\n {\n \"role\": \"system\",\n \"content\": (\n \"Rewrite the user's follow-up into a standalone search query that \"\n \"makes sense without the conversation. Resolve pronouns and implied \"\n \"subjects using the history. Output only the rewritten query.\"\n ),\n },\n {\"role\": \"user\", \"content\": f\"Conversation:\\n{conversation}\\n\\nFollow-up: {follow_up}\"},\n ],\n)\n\nrewritten = response.choices[0].message.content.strip()\nprint(f\"Original: {follow_up}\")\nprint(f\"Rewritten: {rewritten}\")\n```\n\n\nThe history is flattened into the prompt so the model can see what \"it\" refers to, and the system prompt constrains the output to just the query, with no preamble. This rewrite step sits between the user and the retriever: every turn the raw follow-up gets turned into a query that retrieves correctly on its own, which is what makes retrieval work in a real conversation rather than only on isolated questions.\n\n---\n\n# Exercise 2: Rewrite, Then Retrieve\n\nRewriting a follow-up is only worthwhile if it improves retrieval. Feed both the raw follow-up and the rewritten standalone query into the retriever and compare what each pulls back. The pronoun-laden original often retrieves the wrong chunk, while the rewrite finds the right one.\n\n### Requirements\n\n- Rewrite the follow-up into a standalone query using the chat history.\n- Retrieve the best chunk for the raw follow-up and for the rewritten query.\n- Print both results so the improvement is visible.\n\n\n**main.py**\n\n```python\nimport math\nfrom openai import OpenAI\n\nclient = OpenAI() # Uses OPENAI_API_KEY from environment\n\ncorpus = [\n \"Redis persistence uses RDB snapshots and the append-only file (AOF).\",\n \"Redis is an in-memory key-value store often used as a cache.\",\n \"PostgreSQL is a relational database with strong ACID guarantees.\",\n]\n\nhistory = [\n {\"role\": \"user\", \"content\": \"Tell me about Redis.\"},\n {\"role\": \"assistant\", \"content\": \"Redis is an in-memory store often used as a cache.\"},\n]\nfollow_up = \"How does it handle persistence?\"\n\ndef cosine_similarity(a, b):\n dot = sum(x * y for x, y in zip(a, b))\n return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))\n\ndef retrieve(query):\n qv = client.embeddings.create(model=\"text-embedding-3-small\", input=query).data[0].embedding\n cv = [item.embedding for item in client.embeddings.create(model=\"text-embedding-3-small\", input=corpus).data]\n best = max(range(len(corpus)), key=lambda i: cosine_similarity(qv, cv[i]))\n return corpus[best]\n\n# TODO: rewrite the follow-up into a standalone query using the history\n# TODO: retrieve with the raw follow-up AND the rewritten query; print what each returns\n```\n\n\n### Expected Output\n\nThe raw \"how does it handle persistence?\" retrieves poorly without context, while the rewritten \"Redis persistence\" query finds the persistence chunk.\n\n\n```plaintext\nRaw follow-up retrieves: PostgreSQL is a relational database with strong ACID guarantees.\nRewritten query: How does Redis handle persistence?\nRewritten retrieves: Redis persistence uses RDB snapshots and the append-only file (AOF).\n```\n\n\n### Solution\n\n\n**main.py**\n\n```python\nimport math\nfrom openai import OpenAI\n\nclient = OpenAI() # Uses OPENAI_API_KEY from environment\n\ncorpus = [\n \"Redis persistence uses RDB snapshots and the append-only file (AOF).\",\n \"Redis is an in-memory key-value store often used as a cache.\",\n \"PostgreSQL is a relational database with strong ACID guarantees.\",\n]\n\nhistory = [\n {\"role\": \"user\", \"content\": \"Tell me about Redis.\"},\n {\"role\": \"assistant\", \"content\": \"Redis is an in-memory store often used as a cache.\"},\n]\nfollow_up = \"How does it handle persistence?\"\n\ndef cosine_similarity(a, b):\n dot = sum(x * y for x, y in zip(a, b))\n return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))\n\ndef retrieve(query):\n qv = client.embeddings.create(model=\"text-embedding-3-small\", input=query).data[0].embedding\n cv = [item.embedding for item in client.embeddings.create(model=\"text-embedding-3-small\", input=corpus).data]\n best = max(range(len(corpus)), key=lambda i: cosine_similarity(qv, cv[i]))\n return corpus[best]\n\nconversation = \"\\n\".join(f\"{t['role']}: {t['content']}\" for t in history)\nrewrite = client.chat.completions.create(\n model=\"gpt-5.4-mini\",\n messages=[{\"role\": \"system\", \"content\": \"Rewrite the follow-up into a standalone search query. Output only the query.\"},\n {\"role\": \"user\", \"content\": f\"Conversation:\\n{conversation}\\n\\nFollow-up: {follow_up}\"}],\n).choices[0].message.content.strip()\n\nprint(\"Raw follow-up retrieves: \", retrieve(follow_up))\nprint(\"Rewritten query:\", rewrite)\nprint(\"Rewritten retrieves: \", retrieve(rewrite))\n```\n\n\nThe raw follow-up has no subject, so its embedding drifts toward whatever database text is nearby, sometimes the wrong one. Resolving \"it\" to \"Redis\" before embedding aligns the query with the right chunk. Seeing both retrievals side by side makes the value concrete: the rewrite step is not cosmetic, it directly changes which evidence reaches the model, which is why conversational RAG puts it before retrieval on every turn.","pageType":"ai-engineering"}

Get Premium