Calibrating Consistency

By Joanne Sammer Jan 1, 2008

HR Magazine, January 2008Sessions ensure that one manager's 'competent' rating isn't another manager's 'superior.'

Every college graduate remembers how different professors' grading standards were: Some were hard graders, some were easy graders, and some were in the middle. The workplace is often no different. Some managers are hard raters of performance, some easy and some in the middle. According to a study of 5,970 employees reporting to two managers conducted by Personnel Decisions International (PDI), the majority of these employees get inconsistent performance ratings. The study found that employees rated outstanding by one manager were rated lower by their other manager 62 percent of the time. That type of disparity can have major negative implications for employees and organizations.

"It is not OK to have performance rated differently from manager to manager because these decisions impact compensation, development and succession planning," says Dick Grote, a performance management expert and president of Grote Consulting Corp., a management consulting firm in Dallas. Thus, some companies use the practice of calibration to make sure performance appraisals remain consistent across managers. More than simply ensuring fairness in the performance-rating process, conducting calibration meetings allows appraisers to gain insight into who and where their up-and-coming star performers are, to gauge the depth of leadership pipelines, and to ensure that all employees will be evaluated on the same criteria and standards--no matter where they work or to whom they report.

The Calibration Process

Calibration is a face-to-face process; its critical elements are the human interaction and dynamics of calibration meetings. These meetings generally include supervisors and managers responsible for conducting performance appraisals and the executive to whom they report. The meetings should be led by an experienced facilitator, such as the organization's head of human resources, an internal HR person trained as a facilitator or an external consultant. These meetings provide a forum for discussions about individual employees' performances with the goal of making sure managers apply similar standards for all employees. The core of such meetings: discussion of each employee's performance rating and the supervisor's reasons for that rating. The process provides managers with opportunities to discuss the definition of each performance rating and ask questions.

"You may not always see agreement in the room, but you can get alignment," says Michelle Biro, manager of performance at Whirlpool Corp. in Benton Harbor, Mich.

In many cases, managers take what they learn in the calibration session and use it to have constructive meetings with employees about their performances, even when the conversations promise to be difficult ones.

A group can begin the calibration process with outliers--for example, employees who rate a one or a five on a fivepoint scale--because their performances are so clearly good or bad, and create less to discuss. The "toughest calls are likely around employees who are highend threes or low-end fours," says Grote. These decisions require judgment calls to differentiate among employees on the cusp. "This is where calibration pays off, because those individuals … have the potential to be the stars, and the company must identify them accurately," he says.

Many organizations use technology to support the process. Terry Fox, director of HR operations for Direct Energy, a Toronto-based utility, relies on technology that provides real-time access to performance ratings as they happen and tracks where a particular group is with the process. "We focus on leveraging the technology to focus on the people, not to focus on the process," says Fox. "The focus is on the high-value discussion about who the high-performers are and why, and finding ways to share the pool of high-performing employees with peers to help those people develop."

A calibration process can also provide a useful forum for managers in a newly global company to discuss and work through cultural differences to make sure there is a level playing field globally for all employees. To prepare for calibration, HR professionals "should be looking at historical data to spot trends in performance and any history of what may be perceived as bias, such as the fact that most top performers or high-potential employees in a global company are from North America or Europe," says Iona Harding, SPHR, president of Harding Resources, a consultancy in Princeton, N.J. "This way, the facilitator can work to make sure everyone stays focused on the facts and considers cultural differences in how people communicate." Some companies use calibration to validate performance ratings against business results. This can eliminate gamesmanship and resistance to the process, says R.J. Heckman, Ph.D., vice president and general manager for PDI in Minneapolis. He cites a major retailer that relied on the data it collects throughout its operations to drive calibration.

The goal was to develop a pipeline of management talent that the company had been lacking. The company's process focused on whether performance ratings were adequately and accurately reflecting what was happening in individual stores and how each employee was able to influence store-specific results.

Because the company compiles so much data, managers could isolate data affecting store performance and identify individuals delivering results and behaviors yielding positive outcomes. The timing of calibration meetings requires consideration.

Whirlpool Corp. asks managers and supervisors to conduct a pre-appraisal discussion with each employee, focusing on what performance the employee delivered. "The manager will go into the calibration meeting with that information," says Biro.

However, Jim Kochanski, a senior vice president with Sibson Consulting in Raleigh, N.C., says, "The calibration process should be done before all performance ratings and compensation decisions are finalized."

An Eye-Opening Exercise

Managers and supervisors can learn from this calibration process. For example, some managers may find their ratings to be skewed either positively or negatively based on an employee's most recent performance, rather than his or her performance during the entire rating period. The calibration process and the discussions involved can help them see how and why that is happening.

Another benefit: Calibration helps managers and supervisors become better at performance management and employee reviews. In the beginning, Kochanski notes, 20 percent to 30 percent of ratings will require adjustment. But the number of adjustments most likely goes down from year to year as managers and supervisors get better and more efficient at rating performance.

When managers know they will be called upon to defend their ratings in a calibration meeting, they are more likely to be diligent and thorough with their ratings. "There is an element of peer pressure involved because managers will not want to keep making the same mistakes," says Kochanski.

Discussion and feedback from a calibration meeting can also help managers and supervisors better handle potentially contentious employeereview meetings and deliver uncomfortable feedback to underperforming employees. A manager or supervisor is also likely to have more confidence in his or her rating after discussing and defending that rating in a calibration meeting.

Merging Cultures

Consistency in performance ratings becomes particularly important following a merger or acquisition. "Following an acquisition, calibration is virtually mandatory because the two companies will have two different cultures and different expectations of performance," says Grote. "Get people with different backgrounds together to talk through those issues."

When Lawson Software merged with Intentia International in 2006, it grew from 1,400 employees in three countries to 4,000 employees in 30 countries. Calibration was essential to make sure managers were assessing performance consistently on a global basis, according to Kristin Trecker, SPHR, senior vice president of human resources in St. Paul, Minn. This was particularly true because Intentia had not done a lot of performance rating and its employees and managers had to get used to the ratings and what they mean.

"Much of the discussion focuses on those employees who are rated highly because those are really our top performers," says Trecker. "This process was a good way for each company's employees to get to know each other's talented employees." Lawson plans to introduce a calibration process for its senior-level people led by the chief executive officer and one or two other executives. The focus will be to identify key executives, discuss why those individuals deserve that designation and discuss their performance. For the company, this is an opportunity to make sure those employees are being rated appropriately and for senior managers to get to know those individuals and their work better and discuss development opportunities.

Making It Work

Despite benefits, calibration has a downside: "There can be some concern that appraisers will not take ownership of their decisions and, instead, blame the calibration session for their decision if an employee is unhappy" with a rating, says Grote.

A facilitator can go a long way toward minimizing problems. For example, the facilitator must maintain control to keep some managers from having too much influence over calibration decisions. "Some managers can be persuasive and may be able to get their people rated higher," says Grote. "Other managers who are uncomfortable with the process may not go to the mat for their people. The role of the facilitator is to nudge and guide managers to make sure that doesn't happen."

"Over time, calibration influences the whole performance management system," says Harding, a former HR executive for Lucent Technologies and AT&T. "People do a much better job at the beginning of the process by establishing goals and objectives that are linked to the business strategy, as well as monitoring progress and giving feedback at multiple points throughout the year, not just at the end of the year."

Joanne Sammer is a New Jersey-based freelance writer.

Web Extras

SHRM articles
How Kimberly-Clark Ties Pay to Performance (HR Magazine)

Make Merit Pay Matter (SHRM Online Compensation & Benefits Focus Area)

SHRM research :
Performance Management: Getting It Right from the Start

Online sidebar:
Keys to Calibration


Job Finder

Find an HR Job Near You
Post a Job


Find the Right Vendor for Your HR Needs

SHRM’s HR Vendor Directory contains over 10,000 companies

Search & Connect