3 Barriers to the Kirkpatrick Model - And How to Overcome Them
Often called “The Four Levels of Evaluation,” the Kirkpatrick Model has become the most widely accepted model for evaluating the effectiveness of any given training program.
However, while most training experts believe this simple and straightforward evaluation practice is easy to understand and includes the most important measurements of a successful training program, most also find the Kirkpatrick Model almost impossible to implement.
What are the four steps of training evaluation?
Kirkpatrick defines the four steps of training evaluation as Reaction, Learning, Behavior, and Results. In this model, he’s outlining the importance of measuring the following:
1) Reaction: Liked the training and found it relevant to their role in the organization.
2) Learning: Acquired the intended knowledge, skills, and attitude, and have confidence and commitment to apply new knowledge and behaviors on the job.
3) Behaviors: Use the behaviors.
4) Results: Produce the desired organizational results.
Following these steps creates a chain of evidence showing the actual business value of the training.
The Problem: Very Few Get Past Level 1
Those that press on to levels 3 and 4 invest significant time and money with limited results.
At a recent ATD conference, an informal survey of attendees found that 85% of the people in attendance knew the ins and outs of the Kirkpatrick model. Of those who knew the model, 30% had made it to Level 2 (learn the concepts). But less than one percent said they had achieved Level 4.
One person explained how his organization had spent tens of thousands of dollars to reach Level 4 for a single training program. The process was so expensive and time-intensive that they could never do it again.
Therein lies the Kirkpatrick problem—it's easy to understand, it’s nearly unanimous that measurement is important for optimizing training efficacy, but it’s extremely difficult to implement.
The 3 Barriers of the Kirkpatrick Model
Entrepreneurs and business analysts have figured out how to measure and evaluate virtually every other business performance indicator—revenues, ROIs, employee engagement, operating productivity and so on. And yet most organizations have simply accepted that their annual training investment (which averages $1208 per employee per year in U.S. organizations) will continue virtually unproven. Even Dr. Kirkpatrick, the great mind behind the Kirkpatrick model admits that level 3 and 4 measurement can be so difficult for the average organization that “sometimes you may have to be satisfied with data when no evidence exists.”
In our research, we found three significant barriers to implementing the Kirkpatrick Model.
1 - Organizations Lack Research Experts to Collect Training Data (and Collect it Well)
Most organizations do not have the research experts or experience to design and execute data collection. There’s a huge difference between someone writing a few quick survey questions about whether the participants enjoyed training, and designing and implementing a process accurately measuring knowledge acquisition, retention, and long-term application.
2 - Measuring Training Effectiveness Using the Kirkpatrick Model Takes Extra Time and Money
The Kirkpatrick methods to measure the effectiveness of a training program require additional project management, budget, and resources. Since (up to now) there has been no simple, off-the-shelf method to apply the 4-step Kirkpatrick Model to existing training programs, measurement attempts often end up being more expensive than the training itself. Most organizations simply don't have the time, money, and resources to identify long-term training impact.
Unfortunately, without measuring training impact, organizations can get stuck in a cycle of paying thousands of dollars per year (sometimes hundreds of thousands or millions of dollars) on something they’re not even sure works.
3 - Trainees Tend to Have Little Incentive to Participate in Post-Training Evaluation
When organizations attempt post-training evaluation there is typically no incentive for trainees to provide data—and we’re not talking about “Finish this survey and we’ll give you a $10 gift card.” We’re talking about strategic encouragement, skill reinforcement, and personalized support for trainees to report their progress as they apply new skills.
The Kirkpatrick Partners recently released a “New World Kirkpatrick Model” that includes additional points of clarification in each of the 4 steps. Among these is the importance of “processes and systems that reinforce, encourage and reward performance of critical behaviors on the job.” They call this method “Required Drivers” and include it in the Behavior level of the Kirkpatrick Model. The challenge comes in developing the “processes and systems” that will motivate individual participation.
Overcoming the Kirkpatrick Barriers
In our research, we realized developing a solution to overcome these barriers on a business-by-business basis is inefficient, expensive, and will produce spotty results. Most organizations simply don’t have the time, resources and expertise to make the investment pay off.
What if instead there were a technology that could take every existing training program, deliver preconfigured—yet personalized—retention exercises based on Kirkpatrick’s “Required Drivers,” and measure long-term success and application? Then the two final, most critical, and most difficult levels of the Kirkpatrick Model, Behavior and Results, could be turn-key for virtually every organization.
Our training experts, data analysts, and employee development specialists have been hard at work designing just that. Would you like a sneak peek?