Effective Assessments

Page Content

When Nina Brody, head of talent for Take Care Health Systems in Conshohocken, Pa., sought an assessment tool for help in hiring nurses, doctors, medical assistants and others within her health care clinics, she wanted to evaluate candidates’ skills and knowledge and assess them for projected “cultural fit.” How well new hires assimilate into Take Care’s culture—what Brody calls its “special sauce”—plays a crucial role in the retention of about 800 new hires a year in one division where the assessment tool is being used.

“If we have 10 qualified clinical people in front of us, we want to know who will fit best with our culture, because that’s where we tend to experience trouble, not necessarily with someone’s ability to do the job technically,” Brody says.

Those criteria led her to Virtual Job Tryout, an online simulation tool from the Shaker Consulting Group in Cleveland that promises high predictive validity for cultural and team fit as well as assessment of job-specific skills through “day in the life” interactive simulations. Brody says the assessment gives candidates a highly realistic job preview—causing some to self-select out early and others to solidify their commitment—and creates an impression that Take Care is operating at technology’s cutting edge. Take Care recruiters administer the simulation for candidates who pass an initial phone screen.

Brody’s goals are to reduce early-stage turnover, attract more high-quality candidates and leverage technology for more-efficient hiring.

Furthermore, Take Care’s experience reflects many trends and objectives shaping the world of skills assessment today. Whether used for pre-employment screening, training needs analyses or succession planning purposes, properly validated assessments have long been viewed as an important arrow in the performance management quiver.

Assessment Trends

With continued high unemployment rates generating record numbers of job applicants, and more top executives focusing on talent management as a way to build competitive advantage, the ability to quickly, accurately and cost-effectively assess skill levels throughout the employee life cycle has assumed higher priority.

The desire to rapidly separate pretenders from contenders in high-volume hiring situations, and reduce costs early in the selection process, is driving the use of pre-employment assessment technologies. Recent research also suggests that more human resource leaders are using assessments in the post-hire environment by integrating them with talent management systems—for internal recruiting, learning and development, performance management and succession planning—to deliver greater strategic impact.

According to a March study by the Boston-based Aberdeen Group, a market research organization specializing in human capital management, 63 percent of respondents reported using some kind of assessments; of those organizations, 53 percent said they use assessments in the pre- and post-hire stages. The study surveyed HR professionals in 400 organizations representing a variety of industries.

The Aberdeen study found that

86 percent of best-in-class organizations used validated assessments in the pre-hire stage. The research group defines those top companies based on data including performance review ratings, successors identified for key positions and average year-over-year improvement in hiring manager satisfaction.

“Especially for entry-level jobs, hiring managers are looking at assessments to help them more efficiently wade through the large volume of applications they are receiving,” says Mollie Lombardi, senior research analyst for human capital management at Aberdeen.

On the delivery side, new technologies and test formats have transformed assessments, making paper-and-pencil tests an anachronism and introducing more-complex scoring of candidate responses. One such innovation is the virtual simulation assessment, which introduces a game-like feel to testing, mimics real-world job responsibilities and gives candidates a realistic view of what a job may be like. Such assessments can be highly effective for staffing front-line service jobs such as bank teller or call center representative.

Technologies such as Microsoft’s Silverlight, a development platform that integrates multimedia, graphics and interactivity, enable vendors to deliver interactive, robust video for assessment simulations across the web. Development Dimensions International used such technology to develop skills tests for a large automotive manufacturer hiring entry-level assembly workers at a plant. In the past, the Pittsburgh-based assessment firm would have sent a team of assessors to the site to test candidates’ ability to perform complicated assembly tasks. Now, much of that can be done online through simulation followed by computerized scoring.

“The company can test everything from how people tighten a bolt to whether they followed a certain procedure correctly to using a weight-sensitive mat on the floor that, when stepped on at the wrong time, will mark a candidate down in a safety category,” says Scott Erker, Development Dimensions’ senior vice president of selection solutions.

Innovations in assessment and recruiting systems have made life easier for many hiring managers. Now, with some applicant tracking systems, those managers can simply click on a desktop icon that opens to a site featuring competency models, assessment tools, hiring procedures and interviewing guides for most job positions in the organization. If managers haven’t had to hire anyone for six months and their skills are rusty, that just-in-time decision support tool can prove invaluable.

Such systems help HR leaders rest easier knowing managers are using a largely consistent assessment and hiring process across the organization. What can disturb their sleep, however, is the prospect of being sued for assessment practices that are deemed discriminatory or used inconsistently by untrained hiring managers. The biggest threat comes from tests that “might be deemed to have disparate impact on protected classes of employees,” says Steven Anderson, a partner with the law firm of Faegre & Benson in Minneapolis and an employment law expert.

The bottom line: Have your assessment tools reviewed by legal counsel to ensure that they don’t screen out disproportionate numbers of candidates from protected groups, which can include racial or ethnic minorities, people with disabilities, and women, among others.

Here are a few questions about validity to ask vendors of assessment tools:

• When were validity tests done?
• For what jobs was the instrument
  validated?
• What were the results of the validity tests?
• What was the test coefficient?
• What was the sample size?
• Was the test shown to be reliable as well as valid?

Managing Volume

Getting a rapid, accurate and cost-effective gauge on the skill levels of large volumes of job applicants is a major reason organizations invest in pre-employment assessments. At shaving cream maker Barbasol LLC, which is part of Perio Inc., Vice President of Shared Services J.C. Rice used a series of assessment tools from Pittsburgh-based vendor Select International to help fill 30 positions from among 3,000 applicants for a startup manufacturing plant in Ashland, Ohio. The hiring had to be completed within 60 days.

Rice knew the first selection cut would be where efficiency was either captured or squandered, so the initial online assessment featured a pre-screening tool with a number of “knockout” questions for candidates. One such question asked about years of manufacturing experience. Those who made the first cut were then given a “secure fit” assessment that gauged their likelihood of being an attendance risk, safety risk, attitude risk or turnover risk for jobs that included forklift operator, line operator, quality manager, operations manager and more.

“It was extremely helpful to get some of those preliminary questions answered without having to go through 3,000 resumes,” Rice says.

Based on the results of those two assessments and other hiring considerations, candidates were then chosen for behavioral-based interviews; following that, a final simulation-based assessment was used to test for specific manufacturing skills. The cost of the assessment process was about $3,000 per hire for the 30 positions, Rice says, a figure that will drop as the tools are used for future hiring. Although the plant has only been operating since last April, Rice says early results from the new hiring process are encouraging.

“We’ve had no turnover to date, and reports from managers are that most of the new hires are performing very well,” Rice says.

At KeyBank, a Cleveland-based financial services organization, pre-employment assessments, including Shaker’s Virtual Job Tryout simulation, are being used with the goal of reducing 90-day turnover rates and creating more consistency in staffing decisions for tellers and call center associates across the bank’s 14-state operations.

The bank calculated the cost of its pre-hire selection and onboarding at $10,000 per employee for both tellers and call center associates, says Beth Yates, KeyBank’s director of human resources for community banking. “We looked at annualized turnover and 90-day turnover and did the math,” she says. “There was a huge opportunity to take cost out of the staffing process if we could improve retention of new hires and also improve the candidate hiring experience.”

The simulations create an interactive, highly immersive multimedia experience; they mimic key job tasks and test for competencies such as providing client service, adapting to change, supporting team members, following procedures, cross-selling and working efficiently, Yates says.

Before using the assessments, the bank was losing 13 percent of new tellers and call center associates in their first 90 days, Yates says. After implementing the Virtual Job Tryout, that number dropped to 4 percent. “We calculated a $1.7 million cost savings in teller turnover in one year, simply by making better hiring decisions, reducing training costs and increasing quality of hire,” she adds.

Fit Test

Like Take Care Health Systems, more organizations are using assessments to better predict candidates’ cultural fit or to evaluate how well new hires will adapt to distinct company culture, team environments or specific jobs. The rationale: Even an abundance of technical know-how or intellectual horsepower won’t help employees stay longer or be more productive if they don’t enjoy their work or cotton to the team culture.

The Aberdeen study found that “assigning team members based on fit” was among the top five most frequent uses of assessments for best-in-class organizations.

One technique with a high degree of efficacy in predicting cultural fit: asking candidates to state their preferences for certain job conditions. For example, a “fit” question tied to a manufacturing position might be posed this way: “Would you rather have a job where you have to walk significant distances every day, or work in a hot environment every day?” One condition is typically part of the job being applied for, while the other is not.

“By having candidates answer multiple numbers of these types of questions, there is research showing that people who prefer the condition not contained in the job are more likely to turn over faster,” says Kevin Klinvex, co-founder and executive vice president of Select International.

At Take Care, job candidates completing virtual simulations via the Virtual Job Tryout tool receive an overall “fit” score after answering questions such as, “How comfortable are you educating patients about healthy living and wellness?” and “Are you comfortable working in a medical setting that focuses on adult medicine only?” Simulation results are considered along with personal interviews and other evaluation methods in making hiring decisions.

Some recruiters initially resisted using the simulation because it added another step to the recruiting process, Brody says, but with time have bought into its value. “We told them it might negatively impact time-to-fill performance in the beginning, but at the end of the day we felt it would positively impact quality of hire and new-hire retention, which to date it has.”

The simulation was tailored for job roles in Take Care’s employer solutions group through, among other things, hundreds of interviews with incumbent employees. Brody says the company’s investment in the assessment tool, including extensive customization, was about $200,000.

Finding the Leaders

More HR leaders see the value of employing assessments throughout the talent life cycle. The Aberdeen study found, for example, that 42 percent of organizations use assessments because of a perceived shortage of leadership skills and a weak succession pipeline.

“As more organizations face the wave of Baby Boomer retirements, we see them using assessment instruments at a much higher rate for developing succession plans and creating improved leadership bench strength,” says Amy Lewis, practice leader in talent strategy and acquisition for the Human Capital Institute in Washington, D.C.

That’s the case at VSE Corp., an engineering and technical services company in Alexandria, Va., where an initiative designed to create improved leadership bench strength is under way. Using 360-degree and simulation-based assessment instruments from Minneapolis-based Personnel Decisions International Ninth House, the company initially identified 18 top leadership positions and began grooming potential successors.

“The intent is to develop a group of individuals that, upon being sufficiently trained, can move into any of those 18 positions at a moment’s notice,” says Donnelle Moten, VSE’s senior vice president of engineering and logistics. Mid-level and front-line managers were assessed and given individual development plans, to prepare for one day filling leadership roles above them.

Whether the goal is improving cost of hire or quality of hire, or boosting leadership bench strength, assessments have become an indispensible piece of the decision-making puzzle.

The author is a freelance writer and editor in Minneapolis.