Member-only story
Aggregate Bias Audits Considered Harmful
A close reading of the first large-scale study of AI hiring bias, and why every hiring algorithm audit pooled across jobs finds nothing
TL;DR: A study presented at FAccT 2026 examined 4.2 million job applications screened by a single AI hiring vendor across 156 employers, and found that whether the system looks discriminatory depends entirely on how you aggregate the data. Pooled across all positions, every racial group clears the legal four-fifths threshold, which is what the vendor’s own published audit reported. Tested position by position, as U.S. law actually requires, 10.62% of testable positions show adverse impact against Black applicants and 5.32% against Asian applicants, and more than half of Black women applicants encountered at least one such position. The likely mechanism sits in the training setup: each model learns to separate current employees in a role from random database profiles, so it predicts resemblance to incumbents and never observes job performance at all. The study also documents systemic rejection, where applicants turned down everywhere they applied exceed what independent decisions would produce, validated by running the same baseline on 83,000 correspondence-study applications where the mechanism is absent and the baseline fits almost exactly. Job…
