My favorite conversations with training leaders go something like this: “Why do people always blame lack of training when they leave our organization? We are not the problem. After training is when things fall apart.” On the other hand, when I talk to business leaders, it goes more like this: “I do not understand why Jon went through the entire training program flawlessly and still cannot do his job.”
Both conversations are a function of not grasping the meaning of measured performance. As learning professionals, this should be something we can all understand and explain. Here are some tips to help you break it down:
What Is a Performance Measure?
Performance measures, often referred to as key performance indicators (KPIs), are observable measures or competencies that can be quantified over time as objectives, milestones or targets. It is important to know what behavior the learner is expected to perform, and by when and how well they need to perform it at each stage of their learning journey.
What Is Not a Performance Measure?
- Completion tracking.
- Time spent in a tool (i.e., a learning management system)
- Retention in role past 90 days.
None of these items show whether the learner can perform their job. Remember: Performance measurement tells us if a learner can perform their job during training — after training in their role — and whether they (collectively with other learners) affected business results.
Contrary to widespread belief, performance measures are not written assessments covering all the learned material in a course. Learners may know the material, but that does not mean they can perform their job or the required behavior. For example, someone may really like cupcakes. If they eat a cupcake every day, they know that their cholesterol will rise, and they may acquire additional health issues. This knowledge does not keep them from the negative performance of eating a cupcake daily.
How to Measure the Right Things
Though there are many models of measurement in the world of learning, development and performance, the easiest to start with is the Kirkpatrick Model. It looks at the learning program throughout different phases of the learning journey. Before digging deep into this model, it is important to remember that to do this well, measurement cannot be an afterthought. It must be one of the first conversations a program manager or designer has with their stakeholder. Once there is clarity in what a performance measure is, it is easier to determine what good performance looks like. The Kirkpatrick Model is a simple place to begin for professionals who are new to measurement. It is clear, direct, and impactful. While some organizations go straight to the return on investment (ROI) of training, the real impact comes from the Level 3 and Level 4 results. This is so important that we are going to, as Franklin Covey says, “begin with the end in mind.”
Kirkpatrick Level 4: Business Results
Level 4 measurement is the intersection of training and business results. This is where impact is clear. With this level of measurement, one can unequivocally say: Training caused performance that affected business results. In other words, the business results are a direct result of the training program. For this reason, it is important to know, during analysis or discovery, what business outcome your stakeholder is looking for. Without this, your program will never achieve a Level 4 result. Therefore, we begin by knowing the metrics we are aiming for.
Kirkpatrick Level 3: Performance on the Job
The goal of training is to ensure that learners can perform in their role. This is exactly what we measure at Level 3. The goal here is an observable change in behavior. Did the learner apply what they learned when they are on the job? To assess this level of performance well, measurements should be captured at different intervals throughout the learner’s tenure. It is important to look at performance a few weeks post training. It is even more effective to look at expected behaviors six months, nine months and one year later, assessing performance as the learner grows in their role. If done well, the measurement can be based on triggers that indicate whether the learner is up to date for the length of time in the role and can show whether upskilling or reskilling is needed. When measuring Level 3, a rubric is a useful tool, because it shows how well the learner is performing each competency.
Kirkpatrick Level 2: Performance in the Training
While there are different ideas on whether a Level 2 measurement should be a written or practical assessment, the value of a practical assessment or simulation will always net a realistic look at whether the job or skill has been mastered at the level expected during the training class. Level 2 measurements can be taken at intervals during and after the class and applied to the standards required. Rubrics are a useful tool. When using scenarios or simulations for measurement, it’s important to ensure they are as close to the actual tasks performed in the role and in as similar of an environment as possible.
Kirkpatrick Level 1: Engagement
The most important components to gather when measuring Level 1 feedback are:
- Course relevance.
- Learners’ feelings about the experience and logistics.
- Sequencing of materials.
- Learners’ thoughts about the training environment and how confident they are in their ability to apply what they’ve learned.
This is typically done through surveys, interviews or ratings completed by learners. Level 1 data can be gathered throughout the training — by module and/or at the end of the learning program. While some may argue that this is not as important, because it does not net the business results drivers, there is useful information here. If done effectively, Level 1 can tell your team whether the information delivered was sequenced well, whether the environment and activities were conducive to learning and whether the learner feels confident in applying the new information.
While there are many different methods to measure learning effectiveness, the right inputs can ensure that learning teams are focused on the big picture. Each level drives the next until business results are evident. It is important to note that this process takes time to develop. Even if the data does not show positive results at first, an honest look at the process, tools and programs can drive future impacts and create sustainable success for designers, trainers, learners and the business. Don’t fear the data. Embrace it for the future of your team and your organization.