interpretability

The ability to explain or to present an ML model's reasoning in understandable terms to a human. Most linear regression models, for example, are highly interpretable. (You merely need to look at the trained weights for each feature.) Decision forests are also highly interpretable. Some models, however, require sophisticated visualization to become interpretable. You can use the Learning Interpretability Tool (LIT) to interpret ML models.

Practitioners refer to interpretability when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.