Making Decisions About People

Eric J. Sydell, Partner & Vice President, Research and Innovation, Shaker


Eric is a partner and vice president of research and innovation at Shaker. Shaker is a market leader in the creation of engaging, immersive, and evaluative pre-employment assessments called Virtual Job Tryouts(R) that unite three of the most significant trends in HR technology: candidate experience, predictive analytics, and employer branding. Eric created the Research and Development group to drive innovation at Shaker, establishing a way for professional staff to share ideas and explore forward-thinking methods. He leads the production of Standard Virtual Job Tryouts, ready-to-use assessment systems for a wide variety of job types based on Shaker’s award-winning Virtual Job Tryout technology. Passionate about pushing the presumed limits of psychometric measurement, Eric has led the creation of practices to design groundbreaking client solutions. Eric is one of the founders of Shaker and previously served as a consultant at CEB/SHL. Eric holds a Ph.D. in industrial-organizational psychology from the University of Akron and a B.A. in psychology from James Madison University. 

How can we make the best decision possible when choosing whom to hire (or befriend or date or commit ourselves to for one reason or another)? Is there a way to predict a candidate's future behavior or performance on the job? As you may be aware, thousands of companies purport to do exactly this, using a variety of methods from resume keyword matching to background checks to video interviews to assessments to even handwriting analysis—called graphology—to predict a job candidate's future on-the-job success. Graphology has been relatively popular in parts of Europe for many years, despite a lack of research attesting to any stable predictive power. Nevertheless, it would not be unusual to apply for a job in Paris and have your writing quietly analyzed by a graphologist, who may well report that your personality does not fit the requirements of the position.

But graphology is an exception—and an easy target given its lack of scientific basis. How well do the other more accepted methods like interviews and assessments really predict human behavior? Well enough to know with a high level of certainty what a person will do in the future? Almost certainly not. But well enough to make a difference in your hiring process? Definitely.

Science has a very difficult time predicting what specific things individuals will do in the future (if you yourself don't know how long you will keep your next job, how can anyone else?). But, by precisely measuring job-relevant competencies using modern assessment technology, we can identify those candidates more or less likely to be successful in a job, and so, in the aggregate, pre-employment prediction tools can have measurable impact and a large return on investment in the hiring process.

Repeat behavior is easier to predict than one-time or rare events. People often do things that are out of character, but short of actually being able to read a person's thoughts, predicting why or when a person will do something surprising is nearly impossible. 

Our brains are too complicated a mess of neurons and our behavior too emergent to allow for high predictive accuracy. A common example from the hiring world deals with safety violations. Clients often ask us to predict which candidates will be safer on the job, but accidents are infrequent occurrences and difficult to foresee. Although they may stem from stable personality characteristics such as carelessness or impatience, they happen so infrequently that collecting enough data to adequately predict them is nearly impossible.


Job-Specific Hiring Methods

So which of the available hiring methods are useful? And just how reliable are they? The largest body of accepted research in the personnel selection field says that the best predictor of job performance is general mental ability (GMA), what most people think of as IQ or just intelligence. A second high-value addition is the venerable integrity test, which often asks questions about minor workplace transgressions like whether you have used the printer for personal documents or taken home a box of paperclips. Other common hiring tools such as personality tests, realistic job previews and (believe it or not) graphology tend to add little predictive power over these two basic strategies.

The research that has found such high predictive power in GMA and integrity is quite stable, as it is meta-analytic; that is, it combines the results of many separate studies. Such combined data allow us to be fairly certain that overall, across most jobs, GMA and integrity are the best predictors of success.

But there's a rub: GMA has a somewhat controversial history because of its potential for causing adverse impact. As a result, it must be used with caution and tied closely to the requirements of the position. In multimethod assessments, you may choose to weight GMA lower and compensate with noncognitive measures that do not cause adverse impact.

Additionally, the meta-analytic perspective does not consider organization, job, manager and workgroup specifics. Not all jobs require the same skills and abilities. For example, the best predictor of whether a student will make a good theoretical physicist is mental ability, but to be a successful production worker in a factory, the best predictors are typically characteristics like dependability and vigilance.

If you can't simply use GMA and integrity to hire everyone, how do you go about making hiring decisions for a specific job? Essentially, the best approach is to use a hiring process that allows you to measure the key drivers of success for the job. As the meta-analytic findings show, GMA and integrity are likely to be important predictors for most jobs, but how these factors are weighted for a particular position may need to be changed.

Consider this example: A well-known provider of digital communication and entertainment services needed to improve the quality of hire in its call centers. Recruiters and hiring managers were struggling with a high volume of applicants and had no effective tools for identifying those who would be likely to deliver excellent customer service and sell upgrades. In studying the job and incumbent populations, our team found that the job was not technically difficult, but agents needed to work quickly while also delivering top-notch service. They needed to be diligent and focused to handle the high call volume. Based on such criteria, our solution involved a multimethod online assessment and job simulation that was customized to weight the competencies of productivity, service and detail orientation higher than problem solving. Those who scored higher on our assessment showed a 24 percent increase in upselling, had five times better ratings on core values, were 80 percent better on overall performance and were rated as having twice the promotion potential. Following the assessment, candidates could be interviewed, and the interviewers were able to zero in on potential issues using the detailed competency scores on the assessment output report.

At this point in our technological development, the best hiring method tends to involve structured questioning and assessments. In total, how well can these methods predict who will succeed at the job and who will fail? My experience suggests that if you are using well-designed, validated assessments and interviews, and you are able to avoid hiring those who score in the bottom 20 percent, you can realistically expect to see performance gains in the range of 10-40 percent on many indicators of job performance such as sales quotas, customer service ratings, units produced and supervisor ratings. Infrequent occurrences like safety violations are often harder to predict. 

The Candidate's Perspective

When it comes to making decisions about people, it is easy to forget that the applicant is a decision maker, too. For too long we have treated candidates like bacteria to be studied under the microscope rather than thinking beings with a vested interest in finding the best job for them. Recruiting people to apply for jobs often means advertising the job to them, and this mentality leads to leaving out critical information that could dissuade the candidate from continuing in the process. This failure to communicate, in turn, increases costly turnover, as candidates eventually realize they would have made a different decision if they had been better informed up front and leave the job after going through onboarding and training. The solution to this problem is the realistic job preview (RJP), which presents a candid view of the position. RJPs have been used for decades, but usually are, at best, only somewhat realistic job previews. 

Conveying the nature of a job and an organization requires more than listing some pros and cons—it requires presenting the practical and emotional essence of a position. 

Such a preview is best provided through unscripted video interviews of actual incumbents—the type of thing you would see in a documentary or news journalism.

The Near Future

What does the future hold for pre-employment selection? From a scientific standpoint, we expect to see increasingly realistic simulations that will include open-ended responses (typing, speaking). Traditional assessments pose a question and elicit a response that is typically multiple choice. It is straightforward to assign a numeric weight to each response, giving the candidate more points for better answers. But what if we could score what a candidate says or types? This would allow greatly enhanced assessment realism, but existing technology struggles to make sense of such complex, unstructured inputs. However, the field of machine learning is making rapid advances that are helping us do exactly that—automatically score a person's unstructured or freeform responses. Our research shows that we can replicate a human's ratings of another person's responses, and we expect to surpass the predictive accuracy of the human ratings in the near future.

As machine learning improves, it will transform all sectors of the economy, not just employee selection. That voice assistant that lives in your phone or in a speaker on your shelf? Today you can ask it to put paper towels on your shopping list, but tomorrow you may be able to ask it to help you find your next job. And even more interesting is that it may know better than you do which jobs you will like!



Hire the best HR talent or advance your own career.

Discover what’s trending in HR

Search and download FREE white papers from industry experts.

Search and download FREE white papers from industry experts.



HR Daily Newsletter

News, trends and analysis, as well as breaking news alerts, to help HR professionals do their jobs better each business day.