What is a Transformer?
The neural network architecture that revolutionised AI by enabling models to process entire sequences at once.
Pronunciation: /trænsˈfɔːmə/
Transformer explained in plain English
The Transformer is a type of neural network architecture introduced in 2017 that processes data using a mechanism called "attention." Instead of reading text word by word in order (like earlier models), Transformers can look at all words in a sentence simultaneously and determine which ones are most relevant to each other.\n\nThis parallel processing makes Transformers faster to train and more effective at capturing long-range relationships in text, which is why virtually all modern language models are built on this architecture.
Analogy
Imagine reading a sentence where you can instantly see connections between any two words, no matter how far apart they are — like having X-ray vision for language structure. That is what attention allows.
Example
When Google Translate produces a natural-sounding translation, or when a chatbot maintains context across a long conversation, Transformer architecture is doing the heavy lifting behind the scenes.
How is Transformer used?
Transformers are the foundation of GPT, BERT, Claude, Gemini, and virtually every major language model. They are also used in image generation (DALL-E), protein folding (AlphaFold), and speech recognition.
Common misconceptions about Transformer
Transformers are not limited to language — the name refers to the architecture, not transforming one thing into another in a general sense.
History
Introduced in the paper "Attention Is All You Need" (Vaswani et al., 2017) by Google researchers. It replaced recurrent neural networks as the dominant approach for sequence processing.
Related terms
People also read
- LLM
A type of AI model trained on vast amounts of text to understand and generate human language.
- GPT
A family of large language models developed by OpenAI that generate human-like text.
- generative AI
An emerging transformative field with no formal definition.
- AI slop
Output from a generative AI system that favors quantity over quality.
- Attention
A mechanism that lets a model focus on the most relevant parts of its input when producing an output, weighting what matters most in context.
- auto-regressive model
A model that infers a prediction based on its own previous predictions.
- autoencoder
A system that learns to extract the most important information from the input.
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- Chain-of-Thought Prompting
Asking an AI to show its reasoning step by step before giving a final answer, which often improves accuracy on complex tasks.
- Confabulation
When an AI produces a confident, fluent answer that sounds true but is factually wrong — generating plausible language without a reliable link to reality.