AI Can Develop Its Own Hiring Biases, Research Finds

HR leaders have spent years focusing on a familiar problem with artificial intelligence: How do you prevent an AI system from reproducing biases embedded in its training data?

New research suggests that question may not be sufficient.

A study from researchers at Princeton University and the University of Chicago found that large language models (LLMs) can develop novel social biases during repeated hiring decisions.

The finding raises a different challenge for employers adopting increasingly sophisticated AI tools — a system may develop problematic patterns through its own decision-making experience rather than simply inherit bias from historical data or human interaction.

The implication is that AI governance cannot end with a pre-deployment fairness assessment. Organizations also need to understand how AI behaves over time, how decisions are monitored, and what happens when a problematic pattern emerges.

The researchers tested 15 models from providers including OpenAI, Anthropic, DeepSeek, Google, and Meta in a simulated hiring exercise. Candidates belonged to four fictional demographic groups and were equally likely to succeed in every job. The models nevertheless began assigning different groups disproportionately to different occupations based on early hiring outcomes.

The researchers attributed the behavior in part to an explore-exploit trade-off. Rather than continuing to test different possibilities, a model can increasingly rely on patterns that appeared successful earlier in the process. An isolated outcome can therefore become the basis for increasingly broad assumptions.

As the researchers put it, “LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist.” They added that the results show LLMs “are not merely passive mirrors of human social biases but can actively create new ones from experience.”

Beth White, founder and CEO of AI employee support platform MeBeBot, said that the underlying dynamic should not necessarily be viewed as a surprise.

“This is not a new development; it’s a new realization for the mainstream of society of how LLMs and AI models work and are trained,” said White, owner of the Ethical AI for Human Resources LinkedIn group.

She said ongoing human involvement is essential because simply attempting to remove bias from an AI system is not enough.

“Frankly, this is core to how LLMs work and how AI models work,” White said. “That’s why human training of AI tools and training of the models, to remove biases when they surface, is key.”

Event: Blueprint for I&D

Human Oversight Can’t Be an Afterthought

For HR leaders, the practical lesson is not to avoid AI in recruiting. Instead, it is to challenge the idea that an AI recruiting tool can be deployed and then left to operate with minimal supervision.

White said employers need people who understand how AI systems process information and can identify and respond to problematic outputs.

“However, most customers want an ‘easy’ button: turn it on, set it, and forget it. That’s not how AI solutions work,” she said. “There needs to be a human in the loop, who understands how the models process information and more importantly, what you can do about it if you start to detect biases in sourcing, screening, and hiring candidates.”

That human oversight becomes particularly important as employers move toward AI systems capable of making or influencing multiple decisions in sequence. A tool that merely helps summarize resumes presents different risks from an agentic system that evaluates candidates, learns from outcomes, and influences subsequent recommendations.

The study does have key limitations. It used fictional demographic groups and a controlled simulation in which the models received immediate feedback about whether each hiring decision succeeded. That is different from many real-world recruiting systems, which may screen or rank candidates without receiving direct information about whether those candidates ultimately succeed on the job.

“The extent to which this capacity may impact real-world hiring decisions is an open question,” said Michelle Travis, a research professor at the University of San Francisco School of Law. “The study involved fictional demographic groups and controlled data randomization,” she said.

Still, Travis said, the research should prompt employers to take a closer look at their AI practices.

“Despite the study’s limitations, the findings should still be a wake-up call for employers that incorporate AI tools into their hiring process,” she said. “As our understanding of AI bias continues to evolve, the study suggests the value of continued human oversight, particularly when jobs are at stake.”

Specialty Credential: AI+HI

AI Governance as an HR Capability

The research also offers a potentially useful lesson about how employers design AI systems. The researchers found that several interventions — including step-by-step reasoning prompts, increased randomness in outputs, and shorter decision histories to reference — did little to reduce the simulated bias. An explicit diversity objective, instead of a general instruction to be fair, substantially reduced the stereotyping in the scenario.

Travis noted that the finding aligns with other research examining AI-assisted hiring.

“The study also highlights the importance of crafting appropriate instructions for AI tools,” she said.

Organizations should establish clear objectives for how AI is permitted to make recommendations, define appropriate human checkpoints, and periodically test outputs for emerging patterns, not just known forms of bias.

White recommended building those capabilities progressively, beginning with AI literacy and safe-use policies, followed by training on prompts and models and, ultimately, cross-functional evaluation of AI tools by procurement and legal teams.