AlgoMaster Logo

Exercise: Building a Production RAG Pipeline

3 min readUpdated June 22, 2026
Listen to this chapter
Unlock Audio

Exercise 1: Answer a Question with RAG

Wire up the full RAG query path over a small corpus: embed the documents, embed the question, retrieve the top three chunks by cosine similarity, assemble them into a prompt, and have the model answer using only that context.

This is the query pipeline every RAG system runs on each request. The model never sees your whole knowledge base, only the few chunks the retriever selected, so the quality of the answer depends as much on retrieval as on the model. Instructing the model to answer strictly from the provided context is what keeps it from filling gaps with its own training data.

Requirements

  • Embed all documents with text-embedding-3-small and embed the question.
  • Retrieve the top three documents by cosine similarity.
  • Build a prompt that includes only those three chunks as context.
  • Instruct the model to answer using only the context, and to say it does not know if the context does not contain the answer.
  • Print the retrieved chunks and the final answer.
main.py
Loading...

Expected Output

The retriever surfaces the refund chunk, and the model answers from it rather than from general knowledge.

Solution

main.py
Loading...

The pipeline has two halves that fail independently. Retrieval picks the evidence by embedding similarity, and generation turns that evidence into an answer. Pinning the model to the supplied context with the system prompt is what makes the system grounded: when the right chunk is retrieved the answer is correct, and when no chunk covers the question the model abstains instead of inventing a response.

Exercise 2: Multi-Query Retrieval

A single phrasing of a question may miss relevant chunks that use different words. Multi-query retrieval expands the question into several paraphrases, retrieves for each, and unions the results before answering. This widens recall so a vaguely-worded question still finds the right evidence.

Requirements

  • Ask the model to produce two paraphrases of the question.
  • Embed the documents once; for each query variant (original plus paraphrases) retrieve the top chunks.
  • Union the retrieved chunk indices, build context, and answer using only that context.
main.py
Loading...

Expected Output

The paraphrases surface the shipping chunks the original wording might have missed, and the answer is grounded in them.

Solution

main.py
Loading...

Expanding the query trades a cheap extra model call for better recall: each paraphrase probes the index from a different angle, and the union catches chunks any single phrasing would have ranked too low. Deduplicating the indices before building context keeps the prompt tight. This is one of the highest-impact upgrades to a basic RAG pipeline, because retrieval misses, not generation, are the most common cause of wrong answers.