What is automation bias?
The tendency to over-rely on automated systems and ignore or underweight human judgment
automation bias explained in plain English
Automation bias occurs when people put too much trust in automated systems, such as AI or computer algorithms, and fail to consider the potential limitations or errors of these systems. This can lead to poor decision-making and overlooked opportunities for human intervention or correction.
Analogy
Relying solely on a GPS navigation system without considering road signs or asking for directions is like having automation bias. Just as you might miss a detour or construction, automation bias can cause you to miss important details or exceptions that a human would catch.
Example
A doctor might rely too heavily on a medical diagnosis algorithm and fail to consider a patient's unique symptoms or medical history, leading to an incorrect diagnosis.
How is automation bias used?
Automation bias is often discussed in the context of decision-making in fields like healthcare, finance, and transportation, where automated systems are increasingly used to support or make decisions.
Common misconceptions about automation bias
History
The concept of automation bias has been studied in various fields, including psychology and human factors engineering, since the 1980s.
People also read
- attribute
A characteristic or feature of an object or concept
- bias
A systematic error or distortion in a machine learning model's results
- bias (math) or bias term
A constant added to a linear combination of inputs in a machine learning model
- calibration layer
A component in a neural network that adjusts the output to match the true probabilities of a task
- Confabulation
When an AI produces a confident, fluent answer that sounds true but is factually wrong — generating plausible language without a reliable link to reality.
- confirmation bias
The tendency to favor information that confirms existing beliefs or expectations
- counterfactual fairness
A fairness metric in AI that ensures decisions are fair by comparing actual outcomes with hypothetical outcomes where a sensitive attribute is different
- coverage bias
A type of bias that occurs when the data used to train a model does not accurately represent the population or phenomenon being studied
- demographic parity
A fairness metric in machine learning that ensures equal outcomes for different demographic groups
- differential privacy
A method to protect sensitive information in datasets by adding noise to the data