Every semantic search box and retrieval-augmented generation (RAG) system depends on an embedding model: the component that turns text into vectors so that similar meanings end up close together. Pick a weak one and your assistant retrieves the wrong passages, no matter how good the language model is. Here is how to choose.

What an embedding model does

An embedding model maps a piece of text — a query, a sentence, a document chunk — to a list of numbers (a vector). Texts with similar meaning get vectors with high similarity, usually measured by cosine similarity. Retrieval then becomes finding the nearest vectors to the query vector in a vector database.

Two-dimensional plot with points for man, woman, king and queen, with dashed lines linking man to woman and king to queen
A simplified picture of embeddings: words become points in a vector space, and related meanings end up at similar distances and directions from one another. Photo: Singerep / Wikimedia Commons, CC BY-SA 4.0

Sentence-BERT (2019) showed how to train such models efficiently for sentence similarity; today’s models are larger, multilingual and trained on huge collections of query–document pairs.

Use benchmarks to shortlist

The Massive Text Embedding Benchmark (MTEB), introduced in 2022, evaluates models across many tasks — retrieval, classification, clustering, reranking, semantic similarity — and many languages, and maintains a public leaderboard.

Use it wisely:

  • Look at the retrieval scores if you are building search or RAG, not the overall average.
  • Check scores in your languages.
  • Remember that top entries can be very large and slow, and that some may be tuned to the benchmark.

Then test on your own data

  1. Collect 50–100 real questions from users or colleagues.
  2. For each, mark which documents or chunks are correct answers.
  3. Embed your corpus with each candidate model.
  4. Measure how often the right chunk appears in the top 3 or top 5 results (recall@k).
  5. Compare cost and latency at your expected volume.

This small evaluation often reverses leaderboard rankings for specialized domains such as law, medicine or internal jargon.

The trade-offs

Dimensions. Vectors range from a few hundred to several thousand dimensions. More dimensions can capture more nuance but use more storage and memory. Some models support shortened vectors with little quality loss.

Context length. Maximum input length determines how big your chunks can be.

Languages. Multilingual models let a question in Italian find an answer written in English.

Hosted versus open. API models are easy and strong; open models run on your own servers for privacy, cost control and no rate limits.

Speed and cost. You embed your whole corpus once (and again when you switch models), then every query. Large corpora make per-token prices and GPU requirements important.

Domain fit. Code, legal and scientific text may benefit from specialized models or fine-tuning.

Improving retrieval beyond the model

  • Hybrid search: combine embeddings with keyword search (BM25) to catch exact names and codes.
  • Reranking: retrieve 20–50 candidates, then reorder them with a reranker model.
  • Better chunking: see our guide on chunking strategies for RAG.
  • Metadata filters: restrict by date, product or document type before similarity search.

Switching models later

Embeddings from different models are not compatible. Changing models means re-embedding the entire corpus, so store original text and design your pipeline to make reindexing routine.

A quick decision guide

  • English-only documentation search, modest volume: a strong hosted embedding API is the fastest start.
  • Multiple languages: choose a model with high multilingual retrieval scores and test cross-language queries.
  • Sensitive data or very large corpora: an open model hosted on your own infrastructure.
  • Code search: a code-specialized embedding model.
  • Strict latency or edge devices: a small model with short vectors, plus reranking for quality.

What to watch next

  • Multilingual and multimodal embedding models for text and images together.
  • Shorter vectors with near-full quality to cut storage costs.
  • Domain-specific embedding models for code, law and science.
  • Retrieval benchmarks that better reflect real enterprise data.

Key takeaways

  • Embedding models turn text into vectors so similar meanings can be found.
  • Use MTEB retrieval scores in your languages to shortlist candidates.
  • Test recall@k on 50–100 real questions from your own data.
  • Balance quality against dimensions, context length, speed and cost.
  • Hybrid search, reranking and good chunking improve retrieval further.

Frequently asked questions

What is the best embedding model?

It depends on your data, languages and budget. Shortlist with MTEB, then test on your own questions.

Can I mix embeddings from different models?

No. Vectors from different models are not comparable; switching requires re-embedding everything.

Do I need a vector database?

For small collections, simple in-memory search works. Larger or growing corpora benefit from a vector database or vector search in your existing database.