What is a logistic regression?
A type of regression model that predicts a probability.
logistic regression explained in plain English
A type of regression model that predicts a probability. Logistic regression models have the following characteristics: - The label is categorical. The term logistic regression usually refers to binary logistic regression, that is, to a model that calculates probabilities for labels with two possible values. A less common variant, multinomial logistic regression, calculates probabilities for labels with more than two possible values. - The loss function during training is Log Loss. (Multiple Log Loss units can be placed in parallel for labels with more than two possible values.) - The model has a linear architecture, not a deep neural network. However, the remainder of this definition also applies to deep models that predict probabilities for categorical labels. For example, consider a logistic regression model that calculates the probability of an input email being either spam or not spam. During inference, suppose the model predicts 0.72. Therefore, the model is estimating: - A 72% chance of the email being spam. - A 28% chance of the email not being spam. A logistic regression model uses the following two-step architecture: 1. The model generates a raw prediction (y') by applying a linear function of input features. 2. The model uses that raw prediction as input to a sigmoid function, which converts the raw prediction to a value between 0 and 1, exclusive. Like any regression model, a logistic regression model predicts a number. However, this number typically becomes part of a binary classification model as follows: - If the predicted number is greater than the classification threshold, the binary classification model predicts the positive class. - If the predicted number is less than the classification threshold, the binary classification model predicts the negative class. See Logistic regression in Machine Learning Crash Course for more information.
Example
Practitioners refer to logistic regression when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- Bayesian neural network
A probabilistic neural network that accounts for uncertainty in weights and outputs.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- full softmax
Synonym for softmax.
- generative model
Practically speaking, a model that does either of the following: - Creates (generates) new examples from the training dataset.
- gradient boosting
A training algorithm where weak models are trained to iteratively improve the quality (reduce the loss) of a strong model.
- Gradient Descent
The method by which a model gradually improves by making small adjustments after each mistake, moving toward better performance.
- input layer
The layer of a neural network that holds the feature vector.
- minimax loss
A loss function for generative adversarial networks, based on the cross-entropy between the distribution of generated data and real data.
- sigmoid function
A mathematical function that "squishes" an input value into a constrained range, typically 0 to 1 or -1 to +1.