When done effectively, outcomes measured from customer education are impossible to ignore. The most effective way to measure training outcomes is to use a model such as Kirkpatrick’s Learning Evaluation Model.
The Kirkpatrick Model is a popular and widely used framework for evaluating the effectiveness of training. Developed in the 1950s by Donald L. Kirkpatrick, it provides a systematic approach to measuring learning outcomes and, importantly, business results.
The model consists of four levels that represent the different stages of evaluation. Each level builds upon the previous and helps you better measure and showcase impact.
When designing training programs, it’s important to start at those higher levels of the Kirkpatrick Model. What is the business problem you are trying to solve for and how will you know whether your efforts were successful? In this article we’ll review each level of the model, what to measure at each stage and tips on how to pull those metrics.
Level 1: Reaction
At the reaction level, we want to look at:
- Whether customers completed training.
- What their feedback is immediately after completion.
These basic insights become important later in The Kirkpatrick Model to show the business impact of your programs. When gathering learner feedback, use a survey solution. Most of these survey platforms allow you to create dashboards to immediately see the performance of your programs. These dashboards can also be useful for other teams (e.g. customer success).
It’s important to use one scoring system, such as net promoter score (NPS) or customer satisfaction scores (CSAT) to review your progress and track changes over time. Choosing one will also keep your survey shorter and easier for customers to complete, increasing the likelihood of a higher response rate.
In addition to an overall rating, you can add questions to glean insights additional insights. If you are capturing learner data, you can cut up the data in many ways to understand how training is performing for different learner segments, regions, products, etc.
Consider this example:
Using survey data is great way to test drive an idea. After reviewing our different product results against one another we noticed one of them had scores noticeably lower than others. We introduced interactive knowledge checks at the end of our live instructor-led training events and saw a 30% jump on our survey scores over a few months. This demonstrated that our learners enjoyed this level of engagement and the necessity for us to implement it across all our product training programs.
Level 2: Learning
At the second level, we’re working to see if the training program has helped the learner acquire new knowledge and skills. From this level, you can gain insights into whether the programs you provide set your customers up for success.
Assessments can be used to measure knowledge retention. With surveys, you will want a solution that enables you to review multiple data sets. Here is where a learning management system (LMS) can be very useful as a starting point. Within an LMS, not only can you house your training program, which can consist of multiple modalities, but you can add knowledge checks along the way and an assessment at the end. Knowledge checks can help learners ensure they’ve understood key components of the training.
If you deliver live instructor-led training, you can use solutions to gamify this experience making it fun for both you and the customer. Many gamification solutions can allow you to see how learners are progressing, giving you insights into whether knowledge transfer has taken place. You can use this to provide time at the end to recap questions incorrectly answered or adjust your program for the future.
The forgetting curve reminds us that knowledge retention drops off over time if customers are not actively using what they learned in your program. This is where a spaced learning program with an assessment at the end can help to keep the knowledge fresh with your customer.
You can check incorrect answers to adjust your programs for the future and look at multiple data sets, such as region and product, to review differences for future improvements.
We’ve seen reviewing assessments can be very powerful when looking at the later levels of The Kirkpatrick Model. To see if we can support these, we must work backwards from the challenges in the later levels. When we show how we impact the challenges from later levels in the model for the better we can demonstrate the impact we have to the business.
Level 3: Behavior
At this stage, we measure the degree to which participant’s behaviors change as a result of the training, and whether they’re able to apply these new skills and capabilities post-training.
For this level, you should partner with other departments not only to receive feedback for your programs, but also to obtain the support and resources needed to help prove the efficacy of your customer training program.
One way to look at your customers’ behavior change is to review support tickets. Partner with your support team to review the data. With this, you can begin to investigate what your customers do after completing your programs, and additionally, review various data sets, such as region and product, to establish any differences.
In addition to partnering with other departments, you should review customers that’ve been trained versus untrained. This is where you can uncover great value. Do the untrained customers submit more tickets than those who’re trained? If so, it can be a good exercise to quantify this into figures, such as overall hours waited for ticket completion and resulting cost. These figures may use rough estimates of average ticket completion time and average salaries of the roles raising tickets but can be a headline figure to share with your customers to help drive adoption and completion rates.
In addition to support tickets, you can investigate product adoption. You may need to collaborate closely with departments such as professional services, customer success or product to share this information or review insights together.
Consider this example:
After working closely with our support team, we were able to obtain data on support tickets by the user. We reviewed customers we train frequently and ones we rarely train against the average volume of support tickets per user for each of these. As a result, on average the volume of support tickets was one-half lower per user for trained customers. Taking this a step further by looking at average resolution time, we worked out the time our customers saved through engaging with our training to further demonstrate its value.
Level 4: Results
At the final stage of the model, we review tangible business results stemming from the training program. By measuring this, you can prove the value of your programs to higher levels in the business. This can be vital in getting further investment in your team. Do you want more head count on your team, a more powerful LMS or a better instructor-led solution? Your business case can become much easier with not only Level 3 behavior data but concrete business results.
Two ways to get started measuring business results are:
- Product net promotor score (NPS).
- Customer retention.
Trained customers are more likely to be advocates of your product, because they’ve received the knowledge needed to successfully onboard and use your product to its potential. If untrained, gaining this level of knowledge may not be possible or at best take much longer to gain, leading to frustration at the product. And what are the further consequences of this frustration? Customer churn. Frustrated customers are far more likely to look at competing products in the market and leave.
Conclusion
Review different data sets to uncover and track areas of improvement to make your programs more effective, and partner with different departments to get the data you need to help support their efforts and objectives. By utilizing The Kirkpatrick Model to measure training outcomes, you can truly show the value of your training programs to prove that customer education is essential to the success of your business.
