concept drift

A shift in the relationship between features and the label. Over time, concept drift reduces a model's quality. During training, the model learns the relationship between the features and their labels in the training set. If the labels in the training set are good proxies for the real-world, then the model should make good real world predictions. However, due to concept drift, the model's predictions tend to degrade over time. For example, consider a binary classification model that predicts whether or not a certain car model is "fuel efficient." That is, the features could be: - car weight - engine compression - transmission type while the label is either: - fuel efficient - not fuel efficient However, the concept of "fuel efficient car" keeps changing. A car model labeled fuel efficient in 1994 would almost certainly be labeled not fuel efficient in 2024. A model suffering from concept drift tends to make less and less useful predictions over time. Compare and contrast with nonstationarity.

To compensate for concept drift, retrain models faster than the rate of concept drift. For example, if concept drift reduces model precision by a meaningful margin every two months, then retrain your model more frequently than every two months. ---

Practitioners refer to concept drift when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.