Embedding

Definition

A dense numerical representation of data (words, images, etc.) in a continuous vector space. Similar items end up close together in this space. Word embeddings capture meaning — 'king' minus 'man' plus 'woman' equals something close to 'queen.' Essential for search, recommendation, and RAG systems.

How It Works

An embedding is a way of representing something — a word, sentence, image, or any data — as a list of numbers (a vector) that captures its meaning. Similar things end up with similar number patterns. The word "king" and the word "queen" have embeddings that are close together in this numerical space, while "king" and "banana" are far apart.

Embeddings are the secret sauce behind semantic search, recommendation systems, and RAG applications. Instead of matching exact keywords, you can compare embeddings to find conceptually similar content. Search for "how to fix a flat tire" and an embedding-based system finds articles about "changing a punctured wheel" even though no words overlap. This semantic understanding is what makes modern AI applications feel smarter than keyword search.

Creating embeddings requires a trained model. OpenAI's text-embedding-ada-002 and Cohere's embed models are popular choices. The vectors they produce typically have hundreds or thousands of dimensions. These get stored in vector databases (like Pinecone or Weaviate) for fast similarity search. The quality of your embeddings directly determines how well your AI application understands meaning — it's one of the most important infrastructure decisions in building AI products.