Agenda/Training and Development: Needs Assessment and Formative Evaluations

Evaluating Evaluations

Needs Assesment and Formative Evaluations

Tony O'Driscoll, Ed.D., a researcher at the IBM Institute for Advanced Learning in Raleigh, N.C., says that HR spends too many resources on post-training evaluations and not enough on pre-training analysis. He says more time should be spent on determining the purpose of the training and asking respondents about the course during the early stages to alter it based on their feedback, also known as formative evaluation.

"Think about what it is you're trying to measure," he says. "Back-end evaluation is justification. Front-end analysis and formative evaluation are far more fruitful. If someone says, 'We have a problem. We have to do training,'" O'Driscoll recommends asking:

• Who needs to be trained?
• What is it that those people need to do better, more of, or differently than they are today?
• What will it look like when this is achieved?
• What leads you to believe that training will help them achieve this?

O'Driscoll tells a story to illustrate his point. He used to manage a sales training function. One year, fourth quarter sales were flat. Upper management wanted a motivational training course to fire up the sales force. O'Driscoll scheduled a motivational speaker whose program was very popular. However, the next year—after three stellar quarters—the fourth quarter quotas were flat again. So, again, he scheduled a "great motivational guru. He had them hanging off the chandeliers."

When the fourth quarter rolled around the next year and sales were flat again, he says, "I started wondering, could it be that it wasn't a motivational problem?" O'Driscoll examined the profile of the sales force; they were highly trained, well-educated men in their early 30s. He discovered that "we didn't have a training or a motivation issue. We had a lifestyle and compensation issue. These guys were starting to have families. They worked their tails off for the first three quarters, on the road every night," and wanted to be home for the holidays. "No amount of training" would have been able to fix the problem. Therefore, pre-training evaluations can be invaluable in determining what type of training—if any—is required.

If a training program is deemed to be the best course of action, spend some time doing formative evaluations. "Most of the time, trainers ask respondents about the process, not the content," says O'Driscoll. For example: Were the audiovisuals easy to read? Was the course at a convenient time? Did you like the instructor?

Although the feedback to such questions is useful, O'Driscoll says that it would be more beneficial to ask about the employees' reaction to the content. Did you like the content? Did it resonate with you? Is this content that you would like to pursue further? Then, he recommends that you use that information to continue the course with those who are interested in it.

"Use it to maintain a competitive advantage," says O'Driscoll. "If you're holding an introductory course on project management, after two hours, ask who is interested in project management. If 40 people believe this is the driest thing they've ever heard, don't drag those people along kicking and screaming." Continue with those individuals who want to learn about project management, rather than "sheep-dipping everyone," and you will get a greater return on your investment.

Web Extras

• Evaluating Evaluations​​