What is Inference?
The phase when a trained model is actually used — taking new input and producing a prediction or response.
Inference explained in plain English
Analogy
Inference is the moment a musician plays a piece in concert after years of practice. The learning happened in the rehearsal room; the performance is inference.
Example
A hospital runs inference on a trained model to score patient risk in seconds; the model was trained once, but inference runs continuously.
How is Inference used?
Every time you send a message to ChatGPT, every photo your phone tags automatically, and every search result ranked by relevance — that is inference happening in real time.
Common misconceptions about Inference
People also read
- average precision at k
A measure of the accuracy of a model's top k predictions
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- Chain-of-Thought Prompting
Asking an AI to show its reasoning step by step before giving a final answer, which often improves accuracy on complex tasks.
- cross-entropy
A measure of difference between predicted and actual outcomes
- distillation
A process of transferring knowledge from a complex model to a simpler one
- Embedding
A numerical representation of text, images, or other data that captures semantic meaning.
- embedding layer
A layer in a neural network that converts input data into a dense vector representation
- encoder
A component of a neural network that converts input data into a more useful form
- few-shot learning
A type of machine learning that enables models to learn from a small number of examples
- Fine-tuning
The process of further training a pre-trained AI model on specialised data to improve performance on specific tasks.