{"title":"Exercise: Building a Semantic Search Engine","description":null,"content":"# Exercise 1: Search a Corpus by Meaning\n\nYou are given a corpus of ten short documents and a search query. Build the search path end to end: embed the corpus, embed the query, score every document against the query, and print the top three results with their scores.\n\nThis is the same shape as a production search engine. The difference at scale is where the document vectors live (a vector database instead of a Python list) and how you find the nearest ones (an index instead of a full scan). The logic here is the logic that runs underneath both.\n\n### Requirements\n\n- Embed all ten documents. One batched API call is fine.\n- Embed the query.\n- Score each document against the query with cosine similarity.\n- Print the top three documents, most relevant first, each with its score.\n- Embed the corpus once. Do not re-embed it for every query.\n\n\n**main.py**\n\n```python\nimport math\nfrom openai import OpenAI\n\nclient = OpenAI() # Uses the platform-provided API key\n\ncorpus = [\n \"Python is a popular language for data science and scripting.\",\n \"The mitochondria is the powerhouse of the cell.\",\n \"To reset your password, click the forgot password link on the login page.\",\n \"Our refund policy allows returns within 30 days of purchase.\",\n \"Photosynthesis converts sunlight into chemical energy in plants.\",\n \"You can change your billing details under account settings.\",\n \"Java and C++ are compiled languages used for performance-critical systems.\",\n \"The capital of France is Paris, a major center for art and fashion.\",\n \"If your account is locked, contact support to verify your identity.\",\n \"A balanced diet includes proteins, carbohydrates, and healthy fats.\",\n]\n\nquery = \"How do I recover access to my account?\"\n\ndef cosine_similarity(a, b):\n dot = sum(x * y for x, y in zip(a, b))\n mag_a = math.sqrt(sum(x * x for x in a))\n mag_b = math.sqrt(sum(y * y for y in b))\n return dot / (mag_a * mag_b)\n\n# TODO: embed every document in corpus with text-embedding-3-small (embed once)\n\n# TODO: embed the query\n\n# TODO: score each document against the query with cosine_similarity\n\n# TODO: print the top 3 documents and their scores, most relevant first\n```\n\n\n### Expected Output\n\nThe query never says \"password\" or \"locked,\" yet the account-recovery documents should surface at the top. The exact scores vary, but the ranking should look like this.\n\n\n```plaintext\n0.52 If your account is locked, contact support to verify your identity.\n0.47 To reset your password, click the forgot password link on the login page.\n0.39 You can change your billing details under account settings.\n```\n\n\n### Solution\n\n\n**main.py**\n\n```python\nimport math\nfrom openai import OpenAI\n\nclient = OpenAI()\n\ncorpus = [\n \"Python is a popular language for data science and scripting.\",\n \"The mitochondria is the powerhouse of the cell.\",\n \"To reset your password, click the forgot password link on the login page.\",\n \"Our refund policy allows returns within 30 days of purchase.\",\n \"Photosynthesis converts sunlight into chemical energy in plants.\",\n \"You can change your billing details under account settings.\",\n \"Java and C++ are compiled languages used for performance-critical systems.\",\n \"The capital of France is Paris, a major center for art and fashion.\",\n \"If your account is locked, contact support to verify your identity.\",\n \"A balanced diet includes proteins, carbohydrates, and healthy fats.\",\n]\n\nquery = \"How do I recover access to my account?\"\n\ndef cosine_similarity(a, b):\n dot = sum(x * y for x, y in zip(a, b))\n mag_a = math.sqrt(sum(x * x for x in a))\n mag_b = math.sqrt(sum(y * y for y in b))\n return dot / (mag_a * mag_b)\n\n# Embed the corpus once, in a single batched call\ncorpus_response = client.embeddings.create(\n model=\"text-embedding-3-small\",\n input=corpus,\n)\ncorpus_vectors = [item.embedding for item in corpus_response.data]\n\n# Embed the query\nquery_vector = client.embeddings.create(\n model=\"text-embedding-3-small\",\n input=query,\n).data[0].embedding\n\n# Score every document and keep the text alongside the score\nscored = [\n (cosine_similarity(query_vector, corpus_vectors[i]), corpus[i])\n for i in range(len(corpus))\n]\n\n# Print the top 3, most relevant first\nscored.sort(key=lambda pair: pair[0], reverse=True)\nfor score, text in scored[:3]:\n print(f\"{score:.2f} {text}\")\n```\n\n\nThe corpus is embedded in one call before the query, which is what \"embed once\" means in practice: in a real system you would do this at index time and store the vectors, not on every search. Scoring is the same cosine comparison from the earlier exercises, applied across the whole corpus and then sorted to pull out the top three.\n\n---\n\n# Exercise 2: Add a Relevance Threshold\n\nA search engine that always returns its closest match will confidently surface nonsense when nothing is actually relevant. Add a similarity threshold: if even the best match scores below it, return no result instead. This is the abstention behavior that keeps a system honest when the corpus has no answer.\n\n### Requirements\n\n- Embed the corpus once and embed the query.\n- Find the single best match by cosine similarity.\n- If the best score is below `threshold`, return `None`; otherwise return the matched document.\n- Run an on-topic and an off-topic query and print the outcome.\n\n\n**main.py**\n\n```python\nimport math\nfrom openai import OpenAI\n\nclient = OpenAI() # Uses OPENAI_API_KEY from environment\n\ncorpus = [\n \"Python is a popular language for data science and scripting.\",\n \"The mitochondria is the powerhouse of the cell.\",\n \"To reset your password, click the forgot password link on the login page.\",\n \"Our refund policy allows returns within 30 days of purchase.\",\n \"Photosynthesis converts sunlight into chemical energy in plants.\",\n \"You can change your billing details under account settings.\",\n]\n\nthreshold = 0.3\n\ndef cosine_similarity(a, b):\n dot = sum(x * y for x, y in zip(a, b))\n return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))\n\ndef search(query):\n # TODO: embed corpus + query, find the best match by cosine.\n # If the best similarity is below threshold, return None (no good match).\n ...\n\n# TODO: run search() for an on-topic query and an off-topic query, printing the result or \"No good match\"\n```\n\n\n### Expected Output\n\nThe on-topic query returns the account document; the off-topic one falls below the threshold and abstains.\n\n\n```plaintext\nHow do I recover my account?\n -> You can change your billing details under account settings.\nWhat is the best pizza topping?\n -> No good match\n```\n\n\n### Solution\n\n\n**main.py**\n\n```python\nimport math\nfrom openai import OpenAI\n\nclient = OpenAI() # Uses OPENAI_API_KEY from environment\n\ncorpus = [\n \"Python is a popular language for data science and scripting.\",\n \"The mitochondria is the powerhouse of the cell.\",\n \"To reset your password, click the forgot password link on the login page.\",\n \"Our refund policy allows returns within 30 days of purchase.\",\n \"Photosynthesis converts sunlight into chemical energy in plants.\",\n \"You can change your billing details under account settings.\",\n]\n\nthreshold = 0.3\n\ndef cosine_similarity(a, b):\n dot = sum(x * y for x, y in zip(a, b))\n return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))\n\ncorpus_vectors = [item.embedding for item in client.embeddings.create(model=\"text-embedding-3-small\", input=corpus).data]\n\ndef search(query):\n qv = client.embeddings.create(model=\"text-embedding-3-small\", input=query).data[0].embedding\n best_i, best_score = max(((i, cosine_similarity(qv, corpus_vectors[i])) for i in range(len(corpus))), key=lambda p: p[1])\n if best_score < threshold:\n return None\n return corpus[best_i]\n\nfor query in [\"How do I recover my account?\", \"What is the best pizza topping?\"]:\n result = search(query)\n print(f\"{query}\\n -> {result if result else 'No good match'}\")\n```\n\n\nThe threshold turns similarity into a decision: above it, the match is trustworthy enough to return; below it, the honest answer is nothing. Without this gate, the off-topic pizza query would still return its nearest (irrelevant) document, which is how search systems end up giving confident wrong answers. Picking the threshold is empirical, you tune it on real queries, but the abstention pattern is what separates a usable search engine from one that always answers.","pageType":"ai-engineering"}

Get Premium