Machine Learning Advanced 1 min read

What is training-serving skew?

The difference in performance between a machine learning model during training and when it is serving in a real-world application

training-serving skew explained in plain English

Training-serving skew occurs when a model is trained on a specific dataset but performs differently when applied to new, unseen data in a production environment

Analogy

It's like a chef who perfects a recipe in a test kitchen, but when they try to cook it in a busy restaurant, the dish turns out differently due to differences in ingredients, equipment, and cooking conditions

Example

A self-driving car model that is trained on sunny day data may experience training-serving skew when deployed on a rainy day, leading to decreased performance and safety risks

How is training-serving skew used?

Data scientists and engineers use techniques like data augmentation, transfer learning, and online learning to mitigate training-serving skew and improve model performance in real-world applications

Common misconceptions about training-serving skew

Some people believe that training-serving skew is solely due to differences in data, but it can also be caused by differences in model implementation, hardware, or software between training and serving environments

History

The concept of training-serving skew has been recognized since the early days of machine learning, but it has become increasingly important as models are deployed in more complex and dynamic real-world environments

Also known as

concept drift model drift data drift