learning rate
A floating-point number that tells the gradient descent algorithm how strongly to adjust weights and biases on each iteration.
Plain English Explanation
A floating-point number that tells the gradient descent algorithm how strongly to adjust weights and biases on each iteration. For example, a learning rate of 0.3 would adjust weights and biases three times more powerfully than a learning rate of 0.1. Learning rate is a key hyperparameter. If you set the learning rate too low, training will take too long. If you set the learning rate too high, gradient descent often has trouble reaching convergence.
During each iteration, the gradient descent algorithm multiplies the learning rate by the gradient. The resulting product is called the gradient step. --- See Linear regression: Hyperparameters in Machine Learning Crash Course for more information.
How is it used?
Practitioners refer to learning rate when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.