What is logits?
The raw, unnormalized scores produced by a model before applying an activation function
logits explained in plain English
Logits are the output values from a neural network before they are converted into probabilities. They represent the relative likelihood of each possible outcome, but are not yet probabilities themselves.
Analogy
Think of logits like unadjusted exam scores. Just as exam scores need to be scaled and normalized to become percentages, logits need to be passed through an activation function to become probabilities.
Example
In image classification, logits might represent the relative likelihood of an image being a cat, dog, or bird. The softmax function would then be applied to these logits to produce a probability distribution, such as 70% cat, 20% dog, and 10% bird.
How is logits used?
Logits are used as the input to an activation function, such as the softmax function, to produce a probability distribution over all possible outcomes.
Common misconceptions about logits
A common misconception is that logits are the same as probabilities. However, logits are not normalized and do not add up to 1, whereas probabilities do.
History
The term 'logit' was first used in statistics in the 1940s, but its use in machine learning and deep learning has become more widespread in recent years.
People also read
- AUC
A measure of a model's ability to distinguish between positive and negative classes
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- Bayesian neural network
A type of neural network that uses Bayesian inference to make predictions
- Bayesian optimization
A method for finding the best parameters for a machine learning model
- classification threshold
The minimum probability or score required for a model to classify an instance as belonging to a particular class
- configuration
The process of setting up and arranging the components of a system to meet specific requirements
- confusion matrix
A table used to evaluate the performance of a classification model
- cross-entropy
A measure of difference between predicted and actual outcomes
- discriminative model
A type of machine learning model that predicts a target variable based on input data
- embedding layer
A layer in a neural network that converts input data into a dense vector representation