regularization

Any mechanism that reduces overfitting. Popular types of regularization include: - L1 regularization - L2 regularization - dropout regularization - early stopping (this is not a formal regularization method, but can effectively limit overfitting) Regularization can also be defined as the penalty on a model's complexity.

Regularization is counterintuitive. Increasing regularization usually increases training loss, which is confusing because, well, isn't the goal to minimize training loss? Actually, no. The goal isn't to minimize training loss. The goal is to make excellent predictions on real-world examples. Remarkably, even though increasing regularization increases training loss, it usually helps models make better predictions on real-world examples. --- See Overfitting: Model complexity in Machine Learning Crash Course for more information.

Practitioners refer to regularization when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.