AlgoMaster Logo

Exercise: Choosing Embedding Models

3 min readUpdated June 22, 2026
Listen to this chapter
Unlock Audio

Exercise 1: Rank Documents by Relevance

You are given a query and five candidate sentences. Embed all of them with text-embedding-3-small, score each sentence against the query using cosine similarity, and print the sentences from most to least relevant.

This is the ranking test you run to compare candidate models before committing to one. It shows the model's judgment directly: sentences that share meaning with the query should rank near the top even when they share few or no words with it.

Requirements

  • Embed the query and all five sentences. Batching them in one API call is fine.
  • Score each sentence against the query with cosine similarity.
  • Print the sentences in descending order of similarity, each with its score.
  • Do not hardcode the ordering. It must come from the computed scores.

The starter code sets up the client, the data, and a cosine helper. Fill in the embedding and ranking steps.

main.py
Loading...

Expected Output

The exact scores depend on the model, but the order should be stable. The two faucet-related sentences and the plumbing guide should rank above the cake and marathon sentences.

Solution

main.py
Loading...

Putting the query first in the batch lets you split the returned vectors into the query vector and the document vectors with a single slice. The ranking comes entirely from the computed cosine scores, so the same code ranks any query against any document set without changes.

Exercise 2: Compare Two Embedding Models

Choosing a model means comparing candidates on the same data. Rank the documents with two different embedding models and look at whether the orderings agree. When they disagree, that is the signal that the model choice actually matters for your task.

Requirements

  • Write a rank(model_name) helper that embeds the query and documents and returns the documents sorted by similarity.
  • Run it with text-embedding-3-small and text-embedding-3-large.
  • Print both rankings side by side.
main.py
Loading...

Expected Output

Both models put the faucet and plumbing documents on top, but the exact order or score gaps may differ.

Solution

main.py
Loading...

Running both models through the same rank helper isolates the model as the only variable, so any difference in ordering is attributable to the embedding, not the code. On easy cases the two often agree, which tells you the cheaper, smaller model is good enough; where they diverge on harder pairs is where the larger model may earn its extra cost and latency. That comparison, not the dimension count, is how you actually pick.