What is an Embedding?
A numerical representation of text, images, or other data that captures semantic meaning.
Pronunciation: /ɪmˈbedɪŋ/
Embedding explained in plain English
An embedding is a list of numbers (a vector) that represents the meaning of a piece of data — such as a word, sentence, or image. Similar meanings produce similar number patterns, which allows computers to compare and search by meaning rather than exact text matching.\n\nEmbeddings are foundational to semantic search, recommendation systems, and RAG pipelines.
Analogy
Embeddings are like GPS coordinates for meaning. Just as nearby coordinates on a map represent nearby places, similar embeddings in mathematical space represent similar concepts.
Example
When you search "How do I reset my password?" in a help centre, embedding-based search finds articles about "account recovery" and "login issues" even though those exact words were not in your query.
How is Embedding used?
Embeddings power semantic search engines, content recommendation, clustering similar documents, and the retrieval step in RAG systems.
Common misconceptions about Embedding
Embeddings capture statistical patterns, not true understanding. Similar embeddings do not always mean identical meaning in every context.
History
Word embeddings like Word2Vec (2013) and GloVe (2014) pioneered the concept. Modern models produce contextual embeddings where the same word gets different vectors depending on context.
Related terms
People also read
- RAG
A technique that combines AI language models with external knowledge retrieval for more accurate answers.
- LLM
A type of AI model trained on vast amounts of text to understand and generate human language.
- BERT
A model architecture for text representation.
- Character N-gram F-score
A metric to evaluate machine translation models.
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.
- Fine-tuning
The process of further training a pre-trained AI model on specialised data to improve performance on specific tasks.
- generative AI
An emerging transformative field with no formal definition.
- Inference
The phase when a trained model is actually used — taking new input and producing a prediction or response.
- language model
A model that estimates the probability of a token or sequence of tokens occurring in a longer sequence of tokens.
- rotational invariance
In an image classification problem, an algorithm's ability to successfully classify images even when the orientation of the image changes.