noun
Bias in AI refers to the tendency of artificial intelligence systems to produce unfair or prejudiced results, often due to skewed or unrepresentative training data. This can happen when the data used to train AI models reflects historical inequalities, cultural stereotypes, or underrepresentation of certain groups.
For example, facial recognition systems have been shown to perform worse on people with darker skin tones, with error rates up to 34% higher for Black women compared to white men, according to studies like the MIT Media Lab’s 2018 research. Similarly, hiring algorithms may favor male candidates if trained on resumes predominantly from men.
Bias in AI can lead to real-world harm, such as discriminatory lending, biased hiring, or unfair policing. Companies must audit their AI models and use diverse datasets to mitigate these risks. In the EU, the AI Act imposes strict requirements on high-risk AI systems to prevent bias.
A language model might suggest more prestigious job titles for male applicants than female ones when generating resume summaries.