Rectified Linear Unit
An activation function with the following behavior: - If input is negative or zero, then the output is 0.
Plain English Explanation
An activation function with the following behavior: - If input is negative or zero, then the output is 0. - If input is positive, then the output is equal to the input. For example: - If the input is -3, then the output is 0. - If the input is +3, then the output is 3.0. Here is a plot of ReLU: ReLU is a very popular activation function. Despite its simple behavior, ReLU still enables a neural network to learn nonlinear relationships between features and the label.
How is it used?
Practitioners refer to rectified linear unit when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.