What is a Backpropagation?
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
Backpropagation explained in plain English
Backpropagation is the process that tells a neural network which internal settings caused an error and how to adjust them. After making a prediction, the system works backwards through its layers to assign blame and fix mistakes.
Combined with gradient descent, it makes training deep networks practical.
Analogy
Example
Training an image classifier to distinguish cats from dogs depends on backpropagation to update millions of weights after each batch of photos.
How is Backpropagation used?
Backpropagation is built into virtually every deep learning framework and runs silently whenever a neural network is trained — including the models behind modern chatbots and image generators.
Common misconceptions about Backpropagation
People also read
- Gradient Descent
The method by which a model gradually improves by making small adjustments after each mistake, moving toward better performance.
- Bayesian neural network
A probabilistic neural network that accounts for uncertainty in weights and outputs.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- feature engineering
A process that involves the following steps: 1.
- full softmax
Synonym for softmax.
- generative model
Practically speaking, a model that does either of the following: - Creates (generates) new examples from the training dataset.
- gradient boosting
A training algorithm where weak models are trained to iteratively improve the quality (reduce the loss) of a strong model.
- Inference
The phase when a trained model is actually used — taking new input and producing a prediction or response.
- input layer
The layer of a neural network that holds the feature vector.
- layer
A set of neurons in a neural network.