automation bias

When a human decision maker favors recommendations made by an automated decision-making system over information made without automation, even when the automated decision-making system makes errors. See Fairness: Types of bias in Machine Learning Crash Course for more information.

Practitioners refer to automation bias when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.