Embedding

What it means

An embedding is a list of floating-point numbers - often a few hundred to a few thousand of them - that encodes the meaning of an input. A model trained for this task maps text into a space where distance corresponds to semantic similarity, so “cancel my subscription” and “how do I stop being billed” land near each other even though they share almost no words. You compute similarity between two embeddings with a measure like cosine distance.

Embeddings are what make Retrieval-Augmented Generation work. Documents are embedded ahead of time and stored in a vector database. At query time the question is embedded the same way, and the nearest vectors are pulled back as context. The choice of embedding model directly shapes what gets retrieved.

Why it matters

Because retrieval quality caps generation quality, the embedding model is one of the highest leverage decisions in a RAG system. A weak or mismatched model retrieves the wrong chunks, which shows up downstream as low context-precision and ungrounded answers. Embeddings also carry the domain gap problem - a general-purpose model may not separate the fine distinctions that matter in legal, medical, or code search, so teams sometimes fine-tune or swap models to fix retrieval failures that look, at first, like prompt problems.

In practice

When you evaluate a RAG pipeline, changing the embedding model is one of the first experiments worth running, and you measure its effect through the rag-triad rather than by eyeballing results. DeepEval lets you score retrieval quality across a dataset so you can compare two embedding choices on the same questions, and Langfuse traces the retrieved chunks per request so you can inspect what a given model actually surfaced. A practical tip is to keep a fixed evaluation set and re-run it every time you change the embedding model, chunk size, or similarity threshold. See best RAG evaluation tools for platforms that track this over time.