Vendors Seek to Reduce Employers’ Fear of Using AI in Candidate Assessments

HR professionals have been eager to embrace artificial intelligence for tasks such as writing job descriptions, answering employees’ frequently asked questions, and summarizing engagement survey data.

Yet, when it comes to using AI to assess job candidates—particularly in evaluating their personalities, motivational traits, or aptitude—many talent acquisition and HR leaders remain wary of the technology. Concerns about how the use of algorithms, which are a subset of AI, can contribute to biased hiring decisions still run high, driven in part by recent legal actions such as a lawsuit brought against industry vendor Workday alleging the company violated anti-bias laws in using AI to help clients screen job candidates. 

Some industry vendors are addressing those longstanding concerns by offering new AI-driven assessment tools that eliminate aspects of early versions of the technology that raised red flags among industry regulators—such as measuring candidates’ facial expressions or vocal inflections—to help win back the confidence of HR leaders.

Using AI to Evaluate Candidate Language

Research has shown that using AI in any capacity for psychometric assessments of job candidates still gives pause to many HR and recruiting professionals. A recent study by talent management company SHL of 1,600 HR pros found that only 12 percent said they used AI-based assessments when hiring job candidates.

To counter concerns about how AI might create discriminatory hiring outcomes, vendors using the technology in candidate assessments have removed controversial practices such as the use of facial recognition or vocal intonation collected by AI when evaluating candidates. Instead, these providers focus only on applying the technology to evaluate the word choices candidates use during job interviews.

Among those vendors is Talent Select AI, a Milwaukee-based talent assessment company. The vendor uses natural language processing (NLP) in the psychometric assessments it conducts for client organizations to measure job candidates’ personalities, motivational traits, and certain professional competencies.

The technology works by examining the specific words candidates use in transcripts of their live or videotaped job interviews, said Marc Fogel, director of product and industrial-organizational (I/O) psychology for Talent Select AI. The vendor’s assessment approach is based on extensive research in the field of I/O psychology, the expertise of experienced I/O researchers, and a comprehensive validation study that included 66,000 participants, Fogel said.

He said the process features trained “judges” who classify statements made by candidates during job interviews and then place them in predefined categories so they can be assessed statistically for how they help predict success on the job. According to Talent Select AI’s website, the assessment includes the Big Five personality traits, the Great Eight Competencies, and four proprietary motivational traits: grit, enthusiasm, proactiveness, and empathy.

“This is a validated I/O tool that just happens to use AI to create new efficiencies,” Fogel said, referring to NLP’s ability to perform faster and more accurate analyses of candidate word choices during their interviews. “It’s not just the technology revolution enabling us to do this. Along with our own comprehensive validation study, there’s also a large body of research that supports ‘criterion-related validity’ in the use of personality, motivational traits, and professional competencies in predicting job outcomes.”

Fogel said the vendor’s AI model also undergoes regular bias audits that follow U.S. Equal Employment Opportunity Commission (EEOC) regulations as well as the latest AI privacy legislation to ensure unbiased evaluations.

What HR Technology Analysts Think

HR technology analysts and assessment experts say the use of AI to analyze candidates’ word choices in interviews, which other industry vendors such as Aon and HireVue also now do, can help to reduce the chance of bias seeping into assessments.

“There’s a lot of inherent bias in the interviewing process, and using natural language processing to analyze word choices is a way to potentially cut down on the bias that comes from visual appearance or other factors,” said Rebecca Wettemann, CEO and principal analyst of Valoir, a HR advisory firm in Arlington, Va. “Provided the models are trained appropriately, it could be a very effective method” of assessment.

Josh Bersin, an HR industry analyst and founder of the Josh Bersin Academy in Oakland, Calif., said that although most industry vendors who sell AI-based candidate assessments have conducted tests to ensure their tools aren’t biased, those tests often have been limited in other ways.

“Most of the tests I’ve seen are ‘anti-discrimination’ tests, but very few of those recruiting systems have been tested for performance yet,” Bersin said, referring to their validity and reliability in predicting job performance outcomes. “This is still a new market, so the level of rigor adopted in many pre-hire assessments just hasn’t been completed yet. So, most of these tools are EEOC-compliant but not ‘performance validated.’ ”

Ben Eubanks, chief research officer for Lighthouse Research and Advisory, a HR consulting firm in Huntsville, Ala., said the good news for HR and talent acquisition leaders is that standards for validity and reliability in hiring tools haven’t changed even as technology has evolved and AI has become part of the assessment process.

“That means standard selection criteria like job relatedness is still a critical consideration,” Eubanks said. “If a hiring assessment isn’t predictive of job performance, then we might as well be asking for irrelevant information like a candidate’s favorite dessert or grade point average.”

Role of GenAI in Creating Assessments

Generative AI (GenAI) is the newest and most rapidly adopted form of AI, but it has yet to make much impact on psychometric assessments. Experts say the technology can have value in creating certain types of candidate assessments, especially in speeding up the development of first drafts of those tests.

“For example, a company could put a detailed job description and work output into GenAI and ask the GenAI tool to create questions that exemplify what a person would do to be successful in that job,” Bersin said.

Wettemann said she’s seen GenAI be helpful to HR professionals in areas such as assisted authoring.

“Writing questions for job candidate assessments is no exception,” she said. “However, there will still need to be a human in the loop, and assessment questions for very specialized roles may be beyond the capabilities of generative AI without very specific training.”

Dave Zielinski is principal of Skiwood Communications, a business writing and editing firm in Minneapolis.