An embedding model is part of your retrieval infrastructure. Once you choose one, every vector in your index is tied to that model, its dimension setting, its similarity behavior, and sometimes its query/document formatting rules.

That choice affects relevance, latency, storage cost, migration cost, multilingual support, and privacy. A model that works well for short English support tickets may be weak on code search, legal clauses, table-heavy PDFs, or multilingual customer content.

This chapter gives you a practical way to choose: use public benchmarks to make a short list, then test those candidates on your own retrieval task.

Many searches for "best embedding model" lead to the MTEB leaderboard. MTEB stands for Massive Text Embedding Benchmark. It is useful, but it is not a final decision tool. Use it to find candidates, then evaluate those candidates on your own data.

What MTEB Actually Measures

MTEB evaluates embedding models across multiple task categories, each testing a different capability:

For most AI engineering search work, the most important category is Retrieval. This is the category that matters for RAG systems, semantic search, and many recommendation systems. It measures how well a model finds relevant documents for a query. The other categories can matter, but if you are choosing a model for search or RAG, retrieval performance should be your first filter.

A typical MTEB leaderboard entry has these columns:

How to Read It Effectively

Do not sort by overall score and automatically pick the top model. The overall score averages across task categories, including categories that may not matter for your product. A model can rank well overall while being only average for retrieval.

Instead, follow this process:

What MTEB Misses

MTEB is useful, but it has blind spots. If you rely on it alone, you can still choose the wrong model.

Domain-specific performance

MTEB includes many general-purpose datasets, such as web search and question-answering collections. If your data is medical records, legal contracts, source code, support tickets, or internal docs, a model that scores well on public benchmarks may still perform poorly for your domain. There is no substitute for testing on your own data.

Latency and throughput

MTEB does not tell you how fast a model will be in your system. A large model is usually slower to embed text, and higher-dimensional vectors usually cost more to store and search. The leaderboard does not capture those costs.

Cost

Some high-scoring models are API-based and charge per token. Others are open-weight models that you can run yourself, but you still pay for hardware, engineering time, and operations. MTEB does not decide that trade-off for you.

Multilingual performance

Most MTEB benchmarks are English-focused. If your application serves multiple languages, you need to look at MTEB's multilingual subsets separately.

Use MTEB to narrow a large field to a few candidates. Then run a retrieval evaluation with your corpus, your queries, your filters, your chunking, and your latency budget.

Every embedding model outputs vectors of a certain size. That size affects storage, search speed, memory use, and sometimes retrieval quality. Common sizes look like this in practice.

Common Dimension Sizes

Those per-vector sizes look small, but they add up quickly. At 10 million vectors, raw vector storage looks like this:

Going from 384 to 3072 dimensions means 8x more raw vector storage and memory. For 100,000 chunks, that may not matter much. For 100 million chunks, it can become a serious infrastructure cost.

The Quality vs. Efficiency Curve

More dimensions can help, but the relationship is model-specific and not linear. A well-trained 1024-dimensional model can beat a weaker 3072-dimensional model. The better question is cost-adjusted quality: how much recall and precision you get for each dollar, millisecond, and gigabyte.

The quality numbers in the diagram are illustrative. In practice, dimension count, model family, quantization, reranking, and chunking interact. For many systems, 768 to 1536 dimensions is a reasonable range to test first.

Matryoshka Embeddings: Flexible Dimensionality

Some models support flexible dimensionality through Matryoshka-style training. The model is trained so that shorter prefixes of the vector remain useful. You can request or store fewer dimensions and trade some quality for lower memory use and faster search.

OpenAI's text-embedding-3-large supports this pattern through the dimensions parameter. For example:

This gives you flexibility, but it does not remove migration work. A 1024-dimensional index and a 3072-dimensional index are different indexes. If you change the dimension setting after launch, plan a re-embedding and reindexing job.

If you do not want to run model serving infrastructure, API-based models are the simplest path. You send text and receive vectors. The trade-off is ongoing cost, provider dependency, network latency, and data leaving your environment unless you use an approved private deployment option.

OpenAI

OpenAI's text-embedding-3 family is a common baseline:

OpenAI's advantages are a simple API, flexible dimensions, and broad ecosystem support. The downside is that you cannot run these models locally, and changing models or dimensions requires re-embedding the corpus.

Cohere

Cohere's embedding line includes embed-v4.0, which is notable because it supports text, images, and mixed text/image inputs such as PDFs:

One important Cohere feature is the input_type parameter. You tell the model whether you are embedding a search query or a document, and the model uses the right representation for that role:

This asymmetric embedding approach can improve retrieval because queries and documents are different kinds of text. Queries are usually short and intent-driven. Documents are longer and more information-dense. Treating them identically is convenient, but it is not always the right choice.

Voyage AI

Voyage AI is retrieval-focused and offers both general-purpose and domain-specific embedding models:

If you are building code search, legal retrieval, or financial document search, domain-specific models are worth testing against your own data. Do not assume they will win everywhere. Treat them as strong candidates.

Provider Comparison

API models are convenient, but they come with trade-offs: ongoing costs, network latency, data handling constraints, and migration risk. Open-weight models give you more control. You can run them locally or in your own cloud account, tune serving, and avoid per-token API charges.

Common Open-Weight Options

The open model landscape changes quickly. These families are useful starting points, not a permanent ranking:

Smaller models are often good enough and cheaper to serve. Larger embedding models can improve difficult retrieval tasks, but they usually need GPU capacity, batching, and more operational care.

Running Open-Weight Models Locally

The sentence-transformers library makes many open-weight embedding models easy to try:

Output:

The normalize_embeddings=True parameter makes all vectors unit length. With normalized vectors, cosine similarity is equivalent to a dot product, which is convenient for many vector indexes. Some BGE models also expect a specific instruction prefix for retrieval queries:

This is similar in spirit to Cohere's asymmetric approach, but it is implemented through text prefixes rather than a separate API parameter.

When to Choose Open-Weight Models

The decision often comes down to control, cost, and operational capacity. If data residency or privacy is non-negotiable, running an open-weight model inside your environment may be required. If you process large volumes, self-hosting can reduce marginal cost, but the cost does not disappear: you still pay for GPUs or CPUs, engineering time, monitoring, and upgrades.

If volume is moderate and your team does not want to operate model serving, API models are often the better engineering choice. Optimize for total system cost, not only the listed price of the model.

Everything so far has focused on text. Some applications also need embeddings for images, screenshots, scanned documents, charts, or PDFs where the visual layout carries meaning.

When You Need Multimodal Embeddings

Text-only embeddings work when both the query and the searchable content are text. Some applications need to connect text with images or mixed documents:

For these use cases, you need a model that maps text and images into a compatible embedding space. When a text query and a relevant image end up near each other in that space, you can do cross-modal retrieval.

CLIP and Its Descendants

OpenAI's CLIP (Contrastive Language-Image Pre-training) helped popularize multimodal embeddings. It learned a shared space for matching images and text, which made text-to-image and image-to-image retrieval practical.

Today, several multimodal embedding options exist:

Should You Use Multimodal Embeddings?

For most document RAG systems, a strong text embedding model is still the right default. Use multimodal embeddings when the retrieval target includes visual information: product images, screenshots, scanned documents, charts, diagrams, or PDFs where layout and figures carry meaning. If you can extract reliable text and users search with text, benchmark text-only retrieval before adding the complexity of multimodal indexing.

A general-purpose model can struggle with legal clauses, clinical notes, source code, financial filings, or internal terminology. The same word can mean different things in different domains. "Injection" in medicine is not "SQL injection." A "cell" in biology is not a spreadsheet cell.

When General-Purpose Models Fail

The failure mode is often subtle. The system returns plausible results, but not the right ones. A developer searching for "memory leak in connection pool" may get generic memory-management docs instead of the specific code path that fails to release database connections. Semantic similarity is not the same as task relevance.

Your Options for Domain-Specific Embedding

You have three paths, listed from least effort to most:

Option 1: Check whether a credible domain-specific model already exists. Some providers offer models for code, law, finance, biomedical text, or scientific literature. A domain model is not guaranteed to win, but it deserves a place in your evaluation.

Option 2 is fine-tuning. You take a strong general model and train it on domain-specific query/document pairs, hard negatives, or click logs. A few hundred labels can show direction; thousands are usually better. The quality of the negative examples matters as much as the number of positive examples.

Option 3, training from scratch, is rarely the right answer. It requires large training corpora, careful data curation, evaluation infrastructure, and significant compute. Most teams should fine-tune or adapt an existing model before considering it.

How to Evaluate for Your Domain

The only reliable way to know if a model works for your domain is to test it on your domain. A simple evaluation process looks like this:

This takes a few hours of manual work, but it prevents a common mistake: deploying a model that looks strong on public benchmarks and weak on your actual workload.

Public benchmarks narrow the field, but the final decision needs numbers from your own data. The goal is to compare candidate models across the three things that usually decide the choice: retrieval quality, latency, and cost.

Evaluation Metrics

Two metrics are especially useful for embedding evaluation:

Mean Reciprocal Rank (MRR) measures where the first relevant document appears in your ranked results. If the correct document is ranked first, the reciprocal rank is 1. If it is ranked third, the reciprocal rank is 1/3. MRR is the average across all evaluated queries.

Recall@K measures how often a relevant document appears anywhere in the top K results. Recall@10 asks: "Is useful evidence somewhere in the top 10?" This is important for RAG systems because the model can only use documents that retrieval actually returns.

Building a Benchmarking Pipeline

The following script compares multiple embedding models on a small retrieval task:

Using it with sample data looks like this:

Output:

The output above is illustrative. With only a few examples, most models will look good. Differences become visible when you test many queries against a realistic corpus with hard negatives, metadata filters, and the same chunking strategy you plan to use in production.

References