What is an Inference?
The phase when a trained model is actually used — taking new input and producing a prediction or response.
Inference explained in plain English
Analogy
Inference is the moment a musician plays a piece in concert after years of practice. The learning happened in the rehearsal room; the performance is inference.
Example
How is Inference used?
Every time you send a message to ChatGPT, every photo your phone tags automatically, and every search result ranked by relevance — that is inference happening in real time.
Common misconceptions about Inference
Inference is not learning — the model's weights typically stay fixed unless you deliberately retrain or fine-tune.
People also read
- average precision at k
A metric for summarizing a model's performance on a single prompt that generates ranked results, such as a numbered list of book recommendations.
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- Chain-of-Thought Prompting
Asking an AI to show its reasoning step by step before giving a final answer, which often improves accuracy on complex tasks.
- cross-entropy
A generalization of Log Loss to multi-class classification problems.
- distillation
The process of reducing the size of one model (known as the teacher) into a smaller model (known as the student) that emulates the original model's predictions as faithfully as possible.
- Embedding
A numerical representation of text, images, or other data that captures semantic meaning.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.
- few-shot learning
A machine learning approach, often used for object classification, designed to train effective classification models from only a small number of training examples.
- Fine-tuning
The process of further training a pre-trained AI model on specialised data to improve performance on specific tasks.