Hire Intelligence

Use data to discover what really makes a top performer.

By Bill Roberts May 1, 2011

Cosmetics sales associates at Bon-Ton Stores Inc., a department store company that operates in 23 states, routinely had the highest turnover of any sales group in the organization. As a result of a data-driven decision-making initiative, Bon-Ton’s HR professionals devised a more precise pre-employment assessment test, which led to better hires and lower turnover.

“Our turnover in cosmetics is now the lowest among all selling associates,” says Denise M. Domian, senior vice president of HR operations. Another dividend: Associates’ sales-per-hour improved.

Working with a vendor of recruitment products and services, the HR staff studied a sample of the company’s cosmetics associates to determine the skills, knowledge and attitudes that are predictive of longer tenure and higher sales. They mined sales metrics, conducted a survey and correlated the findings. Based on the study results, they rolled out a pre-employment test designed specifically for cosmetics associates. Now, no applicant advances to an interview without a passing grade on the test, and store managers ask certain standard questions during these interviews.

Data-driven decision-making like this is becoming more common among talent managers. Just a few years ago, HR professionals still struggled to understand chi squares and correlation coefficients. Today, vendors offer products and services to help, most organizations have at least some hard data about their employees, and more HR professionals are trained to analyze it.

“The first wave of HR data was about getting metrics,” says Rick Guzzo, a Washington, D.C.-based partner at Mercer, a benefits outsourcing and human capital consulting firm. “Now, HR is moving beyond metrics to analysis. This era of ‘big data’ is not going to be about adding metrics to the dashboard, but about analysis for making decisions.”

‘Big Data,’ Better Decisions

Professionals in business areas other than HR—from finance to supply to quality control—began making data-driven decisions years ago. “HR is in the early era of big data,” Guzzo says.

HR professionals lagged in tapping metrics for several reasons: HR was often the last function to install enterprise software that collects the necessary data, HR leaders only recently began to assume strategic functions—roles that imply greater rigor in human capital decision-making, and many HR professionals lacked a quantitative mind-set.

Guzzo sees many data-based decisions now in sales, call centers and other areas where data make metrics easier to define and measure. “Those jobs have been ones where companies have made some real advances in how they manage and deploy the people they hire that influence the performance metrics,” he says.

Data-driven decisions in hiring are now common. Having “facts about employees and how they perform creates new opportunities for selecting someone,” Guzzo says.

Employers are also mining data post-hire to gain a deeper understanding of turnover and to shape retention strategies. “There is a growing use of data to craft a more nuanced understanding of who is leaving and whether all leavings are regretted,” he explains. Some employers are developing profiles of the attributes the company desires for certain jobs so they can focus retention investments on individuals with those traits.

Smarter Hiring

In 2008, when Bon-Ton’s HR staff began to study turnover among 3,600 cosmetics associates in 275 stores, they identified three areas of concern: selection, onboarding and training. “The assessment of the cosmetics associates was part of a larger strategy, but a major part,” Domian says.

Higher turnover among cosmetics associates contradicted conventional wisdom. These employees have training, expertise and tools from cosmetics vendors to do their jobs. “Their ability to earn more is greater. They make commission while most of our selling associates don’t. You’d think the turnover would be lower,” Domian says.

Bon-Ton, with dual headquarters in York, Pa., and Milwaukee, was using an off-the-shelf pre-employment test for all sales jobs. But “There was a feeling that assessment wasn’t capturing the right people for the beauty advisor position,” she says.

After pre-employment testing, the cosmetic sales group now has the lowest turnover of any sales group.

Working with technology provider Kenexa, based in Wayne, Pa., Bon-Ton’s HR staff launched a study. The project team examined sales metrics, performance review data and employee tenure. It created a sample population for research purposes of 450 current cosmetics associates whose performance and tenure ranged from high to low. A sample of at least 300 was needed for statistical validity.

Key data came from Bon-Ton’s productivity system, which was developed years ago by its HR information systems staff. The system exists mainly to calculate commissions, pulling sales data from the point-of-sale system and employees’ work hours from the payroll system. For the study, the team examined a metric known as sales-per-hour. “Sales-per-hour is a pretty good measurement because it takes out fluctuations” when associates are working at the best or worst selling times, Domian points out.

Anonymously and with no consequences attached, the sample members answered 250 questions drawn from Kenexa’s assessment library. Kenexa analyzed the aggregate results to determine the traits and questions that correlated most highly to the better performers with longer tenure.

Online or in a store, applicants now answer 80 questions on a 20-minute pre-hire assessment test. The test was validated and checked for adverse impact based on race and other factors. Bon-Ton also created 10 standard questions that all store managers ask applicants during an interview.

The Payoff

By the end of 2010, more than 35,000 cosmetics sales associate applicants had taken the test, and about 1,500 had been hired. With a couple of years of data now, the project team recently re-validated and fine-tuned the test. The team also measured the test’s results in achieving the primary goal—reducing turnover.

Average tenure is 12 percent longer for cosmetics associates hired since the new test was implemented, and the cosmetics group now has the lowest turnover of any sales group. Cosmetics associates hired following rollout of the test had 3 percent more sales-per-hour last year than those hired earlier, Domian adds.

“We’re now looking at using the assessment for other commission areas—men’s suits, for example. We’d have to see if the same assessment works for them,” Domian says.

The test focuses on situational judgment, problem-solving and cognitive ability. It includes questions that require math and logic to answer. To Domian’s surprise, cognitive ability is the trait that correlates most closely with lower turnover and higher sales.

“I would have said friendly, good customer service, and someone really into the beauty environment. Those were traits we were targeting,” she says. “But we found the most successful associates and those with most tenure were problem-solvers. They have to take information from the customer about what she wants and needs and solve that problem. This was our ‘aha’ moment.”

The findings came as no surprise to Tiffany Greene-Shortridge, the Kenexa organizational consultant who worked on the project. Cognitive ability tends to predict the capacity to alter selling style to suit the customer and capitalize on each sales opportunity, she explains.

Getting Help

In its behavioral assessment practice, Kenexa offers off-the-shelf and custom tests. About 60 percent of the business is custom, and half of the custom clients already have metrics that can be used for a study like the one conducted at Bon-Ton, Greene-Shortridge says.

Kenexa’s ideal approach to a custom solution: “Use the current employee population, gather performance data on them, do validation studies, and devise the assessment tool,” she says.

Existing metrics also make it easier to conduct return-on-investment studies, as Kenexa recently did for Bon-Ton.

Some clients have less-than-ideal metrics—for example, inconsistent data across stores—but even these can be applied and made useful, Greene-Shortridge says.

When metrics do not exist, Kenexa consultants rely on a “research appraisal,” which is basically a performance review created by managers to acquire baseline data. “The managers will do the performance ratings with the only purpose to develop the assessment tool,” she says.

When testing, do follow-up studies every few years ‘to make sure the job hasn’t changed.’

Behind the scenes at Bon-Ton and elsewhere, Greene-Shortridge starts by conducting focus groups of eight to 10 employees in the population being studied, at least 30 high performers in all. She also holds stakeholder sessions with managers and other leaders. Participants complete a job analysis questionnaire to help her better understand the job being studied, the traits required and the expectations of all parties. This helps her choose questions for an experimental test taken by the sample members.

From start to finish, developing a custom test takes about three months. After about 200 employees have been hired under the new process, Greene-Shortridge recommends doing follow-up studies every few years “to make sure the job hasn’t changed and that we don’t need to adapt the assessment.”

Info vs. Intuition

Kenexa is one of several vendors that offer data-driven decision-making products and services. Others include PeopleAnswers Inc., SuccessFactors and Knowledge Infusion. Vendors use similar methods, hire staff with similar analytics backgrounds and report similar customer experiences.

At first, “Our clients don’t always fully understand the depth to which we are going to explore the issue,” says Ira Grossman, chief operating officer of PeopleAnswers, based in Dallas. “They need some data about performance, and they have to bring to the table their definition of success. Sometimes it is the first time they have had to connect those dots.”

Some clients collect data but never examine it. And some metrics can work at cross purposes. In a call center, for instance, consider the speed of handling a call vs. the quality of caller experience. “You can choose one. Sometimes, the client makes that decision. Or, we can help them come up with a ratio that weighs both,” Grossman says.

About 40 percent of clients, he says, have performance reviews and no metrics. “In retail, for example, they can tell you about sales by store, but not by individual sales associates,” he explains. “That’s typical.” The consultant and the client can develop some metrics for immediate use, and then constantly review, refine and recalibrate the pre-employment test and the metrics.

When PeopleAnswers’ consultants began to work with St. Louis-based Brown Shoe Co. in 2005, the company only had performance reviews, Grossman says.

Kathleen Welter, Brown Shoe’s vice president of talent strategy and leadership continuity, was not involved at the time, but she says business leaders were mainly concerned with retaining sales associates and store managers and their assistants, and identifying the attributes needed for those jobs as the company began to shift to a more customer service-oriented model.

“The selection tool we had was more compliance-based around loss prevention, not the behavioral piece for the service-oriented role,” she explains. Brown Shoe does not use scores from the new test to exclude candidates, but rather as one piece of information for managers. In the first year of the test, Brown Shoe cut turnover in these positions in half and has continued to reduce employee churn.

Grossman says the results are often unexpected. For example, Brown Shoe learned “that higher verbal reasoning was more important in a store manager than an assistant,” he notes.

After completing one of these projects for the first time, vendors and HR professionals say, there is visceral understanding of the power of data-driven decision-making in human resources.

“The bigger ‘aha’ is: ‘Now I understand how performance data ties into this process,’ ” Grossman says, adding that business leaders can control the destiny of their organizations and “can change the performance data.”

The author is technology contributing editor for HR Magazineand is based in Silicon Valley in California.

Web Extras


Job Finder

Find an HR Job Near You
Search Jobs


Find the Right Vendor for Your HR Needs

SHRM’s HR Vendor Directory contains over 10,000 companies

Search & Connect

HR Daily Newsletter

News, trends and analysis, as well as breaking news alerts, to help HR professionals do their jobs better each business day.