AI is increasingly used in hiring processes, even though critics worry that existing biases may be baked into their algorithms. Now, researchers claim that even in the absence of pre-existing biases, AI models can develop brand new social biases.
In a recently published study, a group of researchers from Princeton University and the University of Chicago had a bunch of LLMs complete a hiring game that was previously run with human participants. In the hiring task, participants were asked to assign candidates to specific roles and then received feedback on whether their decision was a successful hire. The candidates were all equally likely to succeed in any given job, but they all belonged to one of four made-up ethnic groups: the Tufa, Aima, Reku, or Weki. When human participants went through this task, the feedback they received caused them to create certain biases against each made-up ethnic group. For example, if they hired a Tufa as a doctor and received negative feedback, they were unlikely to hire another Tufa as a doctor again. The participants even ended up retaining these biases against the made-up ethnic group well after the game ended. When the researchers had LLMs complete this task instead of humans, they found that the bias rates were much higher.
“LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist,” the researchers wrote in the study. “These results reveal that LLMs are not merely passive mirrors of human social biases, but can actively create new ones from experience, raising urgent questions about how these systems will shape societies over time.”
At the heart of this problem is a decision-making principle called explore-exploit tradeoffs. The term describes a pattern of thinking that we, as humans, go through every day when making a decision: should you choose something you never tried before, thus exploring and learning more but potentially coming at a cost to you if it turns out that was a wrong decision, or should you choose what you have chosen and liked before? When the consequences seem high, people often choose to go with what they know and trust (aka exploit) rather than bet on something new (aka explore). Artificial intelligence systems are less incentivized to explore and tend to exhibit reward-maximizing behavior, the researchers say, creating the perfect storm for the creation of stereotypes.
The researchers tested 15 models from providers like OpenAI, Anthropic, DeepSeek, Meta, Google, and Alibaba. Out of all the models, OpenAI’s o3 reasoning model stratified the fake applicants the most severely. Within a family of models, the researchers found that newer, larger models with greater reasoning capabilities produced more biased results.