Questions

AI ethics

What is algorithmic bias?

Algorithmic bias is a system’s tendency to favor or disadvantage people or options because of factors unrelated to what it is supposed to assess. An AI comparing résumés, for example, may favor whoever appears first, even when the candidates are equally qualified.

Engraving: a woman operates a counting machine in front of a panel of dials.
Hollerith’s tabulating machine counted the 1890 US census (Scientific American, 1890). Public domain. Wikimedia Commons

Why it matters

When AI helps select candidates, résumé order, wording, and references to disability can change the ranking. The result can therefore reflect details of presentation, rather than just a candidate’s competence.

What the evidence shows

  1. A study across 70 occupations found that 21 of 22 models preferred the résumé shown first when comparing two equally qualified candidates.Gender and positional biases in LLM-based hiring decisions: evidence from comparative CV/résumé evaluations · David Rozado, PeerJ Computer Science 12:e3628, 2026
  2. A study using GPT-4 found that a résumé with an additional award, scholarship, and position, all related to disability, ranked first in only 15 of 60 comparisons with its version without those additions.Identifying and Improving Disability Bias in GPT-Based Resume Screening · Kate Glazko et al., FAccT 2024, 2024
  3. A study still under review found that rewriting a résumé without changing competence flipped 29.6% of decisions in Llama 3.1 8B and 41.4% in Mistral 7B.Competence-Preserving Resume Perturbations Expose Presentation Sensitivity in LLM Screening · Q. Chen, Y. Xiao, arXiv, em revisão, 2026
  4. Langdon Winner proposed that a technology’s design can embody power relations and favor some people at the expense of others.Do Artifacts Have Politics? · Langdon Winner, 1980

What I would do

I recommend that, before expanding AI use in hiring, people test whether changing only the order or wording changes its choices and keep a person responsible for the decisions.

Why I think so:

  • “AI without evaluation is a bet: measure before you promise.” where it comes from
  • “Technology is not neutral: it carries the choices of those who build it and those who use it.”
  • “A human is always responsible for what AI decides because, as far as we know, only humans are conscious.”

People also ask

  • Can algorithms be biased?
  • What is algorithmic discrimination?
  • Where does AI bias come from?

Read next