Machine Learning Large Language Models Intermediate 2 min read

What is evals?

A unit of measurement for the computational resources required to train an AI model

evals explained in plain English

Evals, short for evaluations, refer to the amount of computational power needed to train and test an AI model. It's a way to measure the complexity and cost of training a model.

Analogy

Think of evals like the number of miles a car can travel on a single tank of gas. Just as a car's mileage affects how far it can go, evals affect how much computational power an AI model needs to learn and improve.

Example

A company developing a new chatbot might use evals to compare the computational resources required to train different AI models. They might find that one model requires 1000 evals to achieve a certain level of accuracy, while another model requires only 500 evals.

How is evals used?

Evals are used to compare the efficiency of different AI models and to determine the resources required to train them. This helps researchers and developers choose the best model for their needs and budget.

Common misconceptions about evals

Some people might think that evals are a direct measure of an AI model's performance or accuracy, but they are actually a measure of the computational resources required to train the model.

History

The concept of evals has been around since the early days of AI research, but it has become more important in recent years as AI models have become more complex and computationally intensive.

Also known as

computational resources training resources model complexity