Is Your AI Hiring Tool Discriminating? Here's How to Find Out

Using artificial intelligence (AI) tools in the hiring process is becoming more common among HR professionals, but rapid adoption and evolution of those tools can pose potential liability challenges for employers. 

That’s why it’s imperative to validate AI-enabled hiring processes, including any selection process, according to Dave Schmidt, principal consultant of DCI Consulting; Bre Timko, senior consultant of DCI Consulting; and Niloy Ray, shareholder of Littler. The three panelists discussed AI in hiring during a panel discussion at SHRM26 in Orlando.

One misunderstanding about validating AI tools is that employers who use AI in hiring but still have a human make the final hire are not immune to potential liability. If your AI tool is screening out or disadvantaging candidates based on protected characteristics, you are opening yourself up to discrimination claims. 

So if a tool is ranking candidates and you only pass the top 20% on for interview, that is a selection, Ray said.

“Every person that you chose not to matriculate up the field because, in part, AI told you that person wasn't as good as someone else, those are the decisions that matter,” he said. “Because the law is looking at it not from the level of who was selected, but from the perspective of who was left.” 

Credential: AI+HI

How to Validate an AI Tool

Many technology vendors will tell you they have validated their products, but if you are not able to see the inner workings of how it makes decisions and comes to conclusions, that opens you up to legal risk, Schmidt said.

“If we got into a legal challenge and someone said, ‘why did I get this score,’ can you actually articulate what were the things that went into that? Do you have the records to show that?” he said. “No amount of arguing that that’s [the vendor’s] secret sauce is going to protect you when there's a subpoena that says you need to explain how this is working.”  

While HR may need cooperation from the vendor for that type of validation, there are ways to check if your AI hiring tool is performing how you would expect. Here are some of them: 

• Criteria-based validation: This compares the scores candidates receive from the tool to how well they actually do the job they are hired for. This can mean comparing candidate scores to things like performance ratings or turnover. The key question is: “Do people who are scoring better on your selection procedure tend to perform better on the job?” Schmidt said.
• Content-oriented: HR can assess what data sources a tool is evaluating — such as what is in a resume and cover letter — and how those sources are being scored. A key question to ask: “Is the content of the selection procedure reflective and representative of important aspects of the job?”
• Construct-based: If you have visibility into how your tool is selecting candidates, you can tell what it is basing its selection on. The key question: “Is the tool measuring what it's supposed to measure and not measuring what it's not supposed to measure?”

How-To Guide: How to Prompt AI to Write Better Job Descriptions

Focus on Fairness

The most important consideration as HR implements AI tools in the hiring process is whether those tools are treating people fairly. Asking these questions of your process, including AI tools, can help: 

• Fairness and equal opportunity: Are all candidates being evaluated under the same conditions, with the same access to accommodations and resources?

• Comparable measurement access: Does your process give every candidate a fair chance to show what they can do without being unduly advantaged or disadvantaged?

• Predictive bias: Is your tool measuring what it’s supposed to measure and not measuring what it should not? Does it predict job outcomes equally well for all demographic groups?

• Adverse impact: Are people with certain protected characteristics passing through your hiring process at significantly lower rates than others? If so, investigate further to assess bias vs. job-related differences.

Vendors will tell you that they have validated their tools, but Schmidt stressed that this is simply a foundation that HR needs to build on with its own data. 

“That doesn’t absolve you of your responsibility. You need to demonstrate that it works in your organization, your jobs, etc.,” he said. 

The biggest takeaway for using AI tools in hiring: validation “is not a one-and-done," Schmidt added. “It’s the type of thing that’s an ongoing accumulation of evidence over time.”