By Jack Edwards, John C. Scott, Nambury S. Raju
2007, 264 pages, Hardcover
SHRMStore Item #61.15004
Order from SHRMStore online or call (800) 444-5006
Be Prepared to Address Potential Excuses for Not Conducting a HR Program Evaluation
Potential Excuse 1: The Resources Required to Conduct a Program Evaluation Are Better Spent on Administering the Program
The resources argument is a common objection to conducting a program evaluation, particularly with large-scale HR programs where significant time and resources have already been invested in the program's development and implementation and where significant accountabilities rest with the program's success. Unless the system is clearly broken or has been challenged, program sponsors often have little desire for investing additional resources to properly evaluate a new or existing HR program. Many times, the program sponsors' perspective is that if resources are available, they should be invested in the program itself.
While program goals are often specified when new HR programs are designed and implemented, there is often less discipline when it comes to establishing the procedures that will ensure that these goals are monitored and evaluated against clear criteria. The internal, and possibly less formalized, evaluation components may therefore be conducted in a haphazard manner or relegated to the back burner as other organizational exigencies and new projects take precedence. Program evaluation needs to be conceptualized as an integral part of an HR program's design and implementation. By incorporating evaluation as a key activity in the implementation plan, with an associated budget, it is treated as a seamless exercise that must be performed in order to optimize the HR program performance.
Throughout this book, our emphasis is on optimizing, rather than maximizing, outcomes regardless of whether we are talking about HR programs or the steps and tools used to evaluate the programs. Without constant vigilance, HR program evaluators can design evaluations and propose HR program improvements that are not feasible on a one-time basis or sustainable over the life of the HR program because the extra resources required to maximize outcomes can result in investments that would outstrip the return that an organization could expect.
To the extent that program evaluation is treated as a separate, independent activity, it may be difficult to convince program sponsors to later allocate resources for this effort, particularly when these resources could be used to better support other aspects of the program (e.g., add a new software module or help fund additional staff). To overcome this potential excuse, program sponsors need to be educated on the value of a program evaluation, in terms of bottom-line impact to the organization. Among other things, this education could include demonstrating whether the program is operating as intended and ensuring ongoing quality improvement.
Potential Excuse 2: Program Effectiveness Is Impossible to Measure
Another potential excuse provided by HR program sponsors for not conducting a program evaluation is the assertion that the impact or effectiveness of the program is too complicated to measure. HR program evaluators must work with HR program staff to provide upper management with answers to questions such as, "How can you tell that the new selection system is resulting in better hires overall or that the revised compensation system has increased employee morale and ultimately retention?" There are clearly challenges involved in measuring the effectiveness of HR programs and human behavior, however most measurement challenges can be addressed through a systematic analysis of the program's goals against well-developed criteria. Program evaluators who are properly trained in research design, assessment methods, and measurement theory and practice will be able to structure an evaluation that focuses on the important variables, constructing evaluation tools for hard-to-measure criteria that must be assessed to answer evaluation questions about program effectiveness and efficiency.
If the evaluation team has been asked to conduct a truly unfeasible evaluation, the team can provide a beneficial service to the organization by identifying an evaluation that is feasible and useful in addressing some of the concerns at hand (Prosavac & Carey, 2003). Other times, the impact of this potential excuse can be allayed by either explaining to top management stakeholders the sorts of assessment tools that might be applied to the particular evaluation or describing the evaluation team's former successes with similar problems and its measurement expertise. We have also found that a give-and-take discussion with the program sponsors can lead to creative ideas on how to measure a program's effectiveness and has the side benefit of engaging an important stakeholder in establishing a collaborative relationship.
Potential Excuse 3: There Are Too Many Variables to Do a Good Study
A related, but slightly different, potential excuse proffered by program sponsors against conducting a program evaluation is that there are just too many outside or contaminating variables to be able to isolate the impact of a specific HR program. This type of argument is frequently heard in sales organizations when discussions occur about performance and compensation programs.
The economy, emerging competition, changing markets, the weather, new legislation, new products to learn and sell, and team vs. individual accountabilities are but some of the variables that can contribute to how effective a HR program is at achieving its goals. While these and other such variables clearly need to be taken into account when evaluating a HR program, there are a number of research designs and statistical analyses that can be conducted to help tease out these variables and isolate the impact of the targeted program. Again, these issues can often be addressed with a discussion of the general evaluation approach that provides examples of how the many variables will be taken into account, along with a solicitation of the program sponsors' input.
Potential Excuse 4: No One Is Asking for an Evaluation, so Why Bother?
This potential excuse may be openly stated by program sponsors, but more frequently it is exhibited as a generalized attitude against conducting an evaluation. This attitude takes the form of "If it ain't broke, don't fix it." The underlying assumptions are that the program left unevaluated will meet its goals as intended and that unless there are complaints, an evaluation leads to nothing more than over-engineering the process. These possibly erroneous assumptions can be short-sighted views that are very challenging to overcome. The key is to identify for the program sponsors the sorts of value that will accrue from conducting an evaluation. Also, the sponsors and evaluation team should come to an understanding about the goals of the HR program evaluation: whether the goals are to demonstrate how well the overall program is working, identify efficiencies that can be realized, both, and/or some other goals.
We have found it is useful in these situations to facilitate a discussion with the program sponsors around how well they believe the goals established for the program are being met and what criteria make sense for affirming these results. This dialog leads to a natural exploration of an evaluation study where the value and return on investment of the evaluation is generally proportionate to the significance of the HR program within the organization.
Potential Excuse 5: "Negative" Results Will Hurt My Program
The implementation of a HR program is often a long process that requires extensive planning, resource leveraging, networking, political maneuvering, and overcoming sometimes substantial organizational resistance. It is therefore not surprising that once a program is finally in place that its sponsor would be reluctant to expose it to the possibility of a negative evaluation. Accepting less-than-positive evaluation findings does not come easy to most people, particularly when those results are made public and could undermine gains that have been achieved in some hard-fought battles.
As with the other potential excuses for not conducting a program evaluation, this argument may be based on the short-sighted view that feedback is harmful and can only result in a losing proposition. This belief may result from the program sponsors' previous experiences within the organization or from the potential for loss of credibility due to the high status or exposure afforded by the target program.
Program evaluation should be positioned as a support tool for the continued usefulness of HR programs. This is a proactive process that is designed to bolster, rather than weaken, the program by providing constructive feedback that maximizes the chance of success over the long run.
Return to Book.