exploding gradient problem
The tendency for gradients in deep neural networks (especially recurrent neural networks) to become surprisingly steep (high).
Plain English Explanation
The tendency for gradients in deep neural networks (especially recurrent neural networks) to become surprisingly steep (high). Steep gradients often cause very large updates to the weights of each node in a deep neural network. Models suffering from the exploding gradient problem become difficult or impossible to train. Gradient clipping can mitigate this problem. Compare to vanishing gradient problem.
How is it used?
Practitioners refer to exploding gradient problem when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.