What is disparate impact?
A situation where a neutral policy or practice has a disproportionately negative effect on a particular group of people
disparate impact explained in plain English
Disparate impact refers to the unintended consequences of a policy or practice that may seem fair on the surface but ends up affecting certain groups of people more harshly than others
Analogy
It's like a one-size-fits-all shoe that may fit most people comfortably, but causes blisters and discomfort for those with larger or smaller feet, even if the shoe itself is not intentionally designed to be discriminatory
Example
A company's AI-powered hiring tool may be found to have a disparate impact on female candidates if it consistently ranks them lower than male candidates with similar qualifications, even if the tool is not explicitly programmed to be sexist
How is disparate impact used?
Disparate impact is often used in the context of AI and machine learning to identify and address biases in algorithms and decision-making systems that may unfairly affect certain groups, such as racial or ethnic minorities
Common misconceptions about disparate impact
Some people may mistakenly believe that disparate impact requires intentional discrimination, but it can occur even if the policy or practice is neutral on its face
History
The concept of disparate impact originated in the 1970s in the United States as a way to address unintentional discrimination in employment practices
People also read
- A/B testing
A method of comparing two versions of a product or service to determine which one performs better
- ablation
A technique used to remove or disable parts of a machine learning model to understand their importance
- accuracy
The degree to which a model's predictions match the actual outcomes
- activation function
A mathematical function that introduces non-linearity into a neural network model
- active learning
A machine learning approach where the model actively selects the most informative data to learn from
- adaptation
The process of adjusting to new or changing conditions
- agglomerative clustering
A type of hierarchical clustering that groups similar data points together
- anomaly detection
The process of identifying data points that do not conform to expected patterns or behaviors
- area under the PR curve
A measure of a model's performance in classification tasks
- area under the ROC curve
A measure of a model's ability to distinguish between positive and negative classes