All too often, we measure learning success completion or engagement. This information is good if we want happy learners, but it does not tell us whether the performance bar moved. Part of the problem is that when we design training, we use a hybrid of design models and overlook the element of good analysis. But without having that element in place from the beginning, we will never know whether we hit the bar, raised the bar or were hit with the bar.
Stakeholders, learning leaders, trainers and instructional designers must be on the same page about what good performance looks like, but being on the same page is not the end of the story. We need to focus more on not just designing, implementing and evaluating (D.I.E.). Good measurement begins at the beginning of the program. When it doesn’t, there is a discovery gap, and your analysis may not tell the whole story.
Here are four tips for measuring learning for impact:
1. Evolve Your Data Approach
Start with the cost of the learning to determine what remains stable within the context of spending. Unless there is significant change happening in your organization, this amount should be relatively stable. Items to consider include:
- Project details by hour for scoping, designers, design, tools, translation, pictures and graphics, etc.
- Program implementation costs, such as trainer and learner hours per class and by regional area, if needed.
- Travel expenses, including food, lodging and per diems.
- Any additional miscellaneous expenses that may be related to the work or your industry.
2. Look at How the Learning Will Achieve Business Results and Impact
This examination includes following career movement laterally and/or vertically as well as looking at the business returns and performance impact. As learning and development (L&D) programs are more successful, business revenue will rise accordingly, people will perform more effectively, and there may also be positive attrition as people are promoted to new roles inside or outside the organization.
3. Know What the Expected Result Should Be
What problem is the learning attempting to solve? If you or the business does not know the answer to this question, all the training in the world will not help. What aren’t people doing? How do you know? What do they need to do, how well do they need to do it, and how can you leverage data to show that it is happening or not happening? What data currently exists about performance?
Gaining this information requires asking good questions up front. Plan for this conversation with your stakeholder. If you do not know what questions to ask, no amount of data can help. If you want your learning organization to make an impact, you must have the right data to understand needs and determine whether performance has changed based on the solutions you implemented. Your business is looking for continuous improvement as well as speed in the data analysis and the performance story behind it.
4. Make Sure Your Data Is Reliable and Unbiased
Pressure-test your questions to avoid response bias:
- Keep questions simple and short. If a question is too long, your respondent may skip it.
- Avoid leading questions that begin with phrases like, “Are you …,” which encourage learners to respond in a certain way. (For example, “How satisfied are you with …” indicates that the learners are satisfied.)
- Do not ask questions that require people to break down knowledge of a concept. Instead, use simple language.
- Avoid yes/no or true/false questions; instead, use an interval scale question (a numerical rating).
- Put any personal questions at the end of the survey, this way, if the respondent drops out, you still have the data you need.
- Continuously track the metrics of your data, including average scores for questions, topic responses and drop rates.
Once you’ve identified the necessary data and accessed it, you can plan and complete your strategy. This model is not one size fits all; every organization is different, and you must scale your questions and answers to support the need. Many organizations use the Kirkpatrick Model; each level of the model may not be right for every program, but if you are looking for ways to measure behavior and business results, it is a popular method.
The main idea here is that you must know what you want to impact and how you are going to measure it before designing, developing and implementing programs, regardless of the design model you use. Measurement must be a forethought, not an afterthought.
Want to learn more on this topic? Sign up for the virtual Training Industry Conference & Expo, and attend Loren and Jon’s live session.