AI Hiring Algorithms Show Racial Bias, Study Reveals

AI-driven hiring processes promise efficiency, but at what cost? A recent study led by Stanford researchers reveals troubling racial biases in algorithmic hiring decisions. The investigation focused on a hiring vendor, pymetrics, acquired by Harver in 2022. Their algorithm, despite its promise to de-bias, shows significant disparities against Black and Asian job applicants.

Unpacking the Data

The team, consisting of Rishi Bommasani, Sarah H. Bana, Kathleen A. Creel, Dan Jurafsky, and Percy Liang, scrutinized a pymetrics dataset from December 2018 to December 2022. With over four million job applications analyzed, the study spanned 11 industries, covering 156 employers with a combined revenue of $225 billion. The platform's recommendation process favored only about 58.2% of applicants per position, leaving employers to typically ignore those not endorsed by the algorithm.

According to the Equal Employment Opportunity Commission’s "four-fifths rule," disparities become concerning when a group's selection rate falls below 80% of the highest group's rate. The study found that 26% of Black applicants and 15% of Asian applicants faced discrimination from the AI system, compared to their White counterparts. If fairness prevailed, approximately 40,000 more candidates from these marginalized groups could have advanced in the hiring process.

Algorithmic Monoculture: A New Threat

The issue compounds when job seekers apply to multiple companies using the same algorithmic system. The study found that candidates applying to four positions were rejected by all, more often than if each company used a distinct hiring method. This monoculture effect, where a single algorithm influences multiple hiring decisions, leads to systemic rejection patterns that aren't present in traditional hiring studies.

Is this what we've signed up for with AI in recruitment? The so-called efficiency masks an insidious bias, as algorithms tend to use proxies for demographic data, like zip codes or schools, which perpetuates discrimination. Decentralized compute sounds great until you benchmark the latency, and here we're witnessing a similar pattern: impressive on paper, but deeply flawed in practice.

The Way Forward

This finding isn't isolated. Previous studies echo that AI systems can discriminate even without explicit demographic data. Pymetrics' own research claimed their algorithm complied with EEOC standards, yet the Stanford study highlights the flaws in averaging recommendations. The real discrimination is masked when you don't analyze each job separately. Imagine the AI preferring Black applicants for warehouse roles but not for finance. Aggregating results hides these disparities.

Slapping a model on a GPU rental isn't a convergence thesis, and neither is relying on a one-size-fits-all AI for hiring. If AI can hold a wallet, who writes the risk model? As this study indicates, true fairness in AI hiring remains elusive. Stakeholders need to address these biases head-on, demanding transparency in AI systems. The intersection is real. Ninety percent of the projects aren't, and pymetrics is a glaring example.