How to Evaluate Hiring Assessments

​When faced with a hiring decision, HR professionals and managers have to consider everything they know about the applicants. But that might not be enough information to make a choice. To get more information and add objectivity to the decision-making process, many organizations use assessments. 

"When these tools are used correctly, they're tremendously valuable," said Eric Sydell, Ph.D., an industrial-organizational psychologist and the chief innovation officer at Modern Hire, a hiring platform. "There's a level of objective assessment about a person that can be very predictive."

HR professionals and hiring managers don't have the ability to make accurate predictions in the same way assessments can. "Our brains don't work that way," he said. 

But buyer beware, Sydell warned, "There are a lot of tools out there that sound great on the surface" but fail to deliver valid and reliable results.

[SHRM members-only toolkit: Recruiting Internally and Externally]

Multiple Options Present Tough Choices

Ryne Sherman, chief science officer at Hogan Assessment Systems in Tulsa, Okla., said he suspects that most large corporations are probably using reliable and valid assessments, while smaller businesses may not be. Unfortunately, he said, "With this industry, there is no regulating body at all—literally anybody can make an assessment tool and start selling it with no background and no science put into it whatsoever."

Perhaps because of the open nature of the field, there are a lot of tools to choose from and many of them are complex, making the selection of one of them a potentially confusing—and even risky—decision to make.

Must-Haves for Effective Assessment Tools

Ryan Lahti, Ph.D., is an industrial-organizational psychologist and the founder and managing principal of OrgLeader, a management consultancy in Newport Beach, Calif. He uses a variety of assessment tools in his work. There are many factors to be considered when evaluating an assessment tool, he said, but the three key ones are validity, reliability and the population that was used to develop it.

Validity deals with how accurately the tool measures the concepts it claims to measure. Lahti pointed to three forms of validity:

• Content validity indicates how well the tool measures a representative sample of the subject of interest. At a minimum, he said, you want a tool that has content validity.
• Criterion validity indicates how well the tool correlates with an established measure or outcome—for example, correlation to strong performance ratings.
• Construct validity indicates how well the tool measures a concept or trait—for example, conscientiousness.

Reliability is a measure of how consistently the tool measures issues of interest. If you were to give the same assessment to the same candidate more than once, how similar would the results be?

Finally, the population used to develop the tool is an important consideration and should be the same as the population being assessed. "For example, you would not want a tool developed on an adolescent population to be used to assess working adults," Lahti said.

Sherman offered the Minnesota Multiphasic Personality Inventory (MMPI) test as an example. The popular assessment tool used by organizations to screen candidates was designed for diagnosis and treatment of mental health conditions, he said. That can be a risky tool to use for assessing the potential of job applicants.

What to Watch Out For

As HR leaders consider various assessment options, they need to thoroughly evaluate whether the assessments they're considering incorporate the must-haves. Look out for companies that don't publish information on validity, reliability and the population used to develop the assessment.

Some companies, Lahti said, will say that their tools are used by a lot of Fortune 500 companies.

"While this argument shows they have good sales and marketing departments, it does not prove the companies have sound assessment tools," he said.

Sy Islam, Ph.D., an associate professor at the State University of New York at Farmingdale and a vice president at Talent Metrics consulting firm in Melville, N.Y., said employers should ask test companies to show their worth. "Vendors should be able to provide you with a validity coefficient, which is a statistic—a correlation coefficient—that indicates how much predictive validity the assessment has," he said. He warned against accepting "black box" explanations like "the tool is proprietary and cannot be explained." The ability to support your assessment could become an issue if your company becomes involved in a lawsuit, he said.

"What you don't want to do is rely on high-level summary statements, marketing statements or hype that is generated by these companies," Sydell said. "There are a lot of different buzzwords and catchphrases that vendors will throw out there. It's really important to look beyond that and dig below the surface." And, while he says you don't need a Ph.D. to do that, it is a good idea to seek help from someone who is familiar with these types of assessments and can help evaluate their efficacy.

"I would strongly advise finding a local industrial and organizational psychologist who can evaluate different vendors and talk to you about best practices," Sherman said. The proper assessment and selection of candidates is just too important, and potentially risky, to cut corners.

Lin Grensing-Pophal is a freelance writer in Chippewa Falls, Wis.