If the only audience for your evaluation data and reports are your executives and L&D staff, you’re missing your most important audience.
Often, L&D professionals point to executive leadership as their primary audience. However, a 2015 study by Brandon Hall found that over 60% of L&D directors feel no pressure from upper management to gather learning metrics. This limited interest in evaluations harms value in two ways. First, it limits the ability to take action based on the evaluation results. Second, it makes it more difficult to acquire good performance data.
What if we were to rethink the primary audience and purpose of evaluations?
The Key Stakeholders for Learning Evaluation: The Learners’ Managers
The primary purpose of L&D is to create behavior change, and it is the learner’s manager who has the greatest impact on whether learning is used and applied on the job. Our research has shown that manager involvement in learning and learning transfer can increase impact by 40% or more.
Unfortunately, most learning initiatives do a poor job of enabling managers to support learning. In most cases, managers receive a summary of the skills learned or nothing at all. As a result, managers are in a poor position to reinforce and coach new skills. For learning to transfer to the workplace, managers need the knowledge and skills to effectively coach.
Impact evaluations can be an effective method to give managers what they need and also hold them accountable for taking action. By focusing learning evaluation on the data and reporting needs of the manager, you accomplish two things: First, you increase the amount of learning that is applied and used. Our research shows that only 30 to 35% of learning is used back on the job; simply involving the manager can more than double that amount. Second, you gain access to important metrics (managers’ observations of performance), which increases the impact, credibility and validity of the evaluations.
While far from a traditional approach to evaluation, by focusing on the learner’s manager as the key stakeholder, you not only improve the validity and acceptance of the evaluation results, but you also improve the application and use of skills and, ultimately, the performance of the learners.
A Manager-focused Evaluation Process
Creating a manager-focused evaluation process requires an organization to not only rethink evaluations but to rethink learning design. The following steps are critical to gaining maximum value from impact evaluations.
- Design for a Learning Journey, not a Training Event
Managers need to be active throughout the learning process. If the learning process consists of only a one- to two-day training event or a single e-learning event, it is impossible to have managers involved in a meaningful way. Designing it as a learning journey with a blend of different delivery modalities (web meetings, microlearning, classroom learning, etc.) will allow managers to support and coach learners throughout the process.
- Identify Predictive Failure Indicators (PFIs)
The most valuable manager coaching comes when the possibility of failure to apply learning is highest. Some skills are harder to use, and there are times in the learning process when fatigue or forgetting is more common. These predictive failure indicators (PFIs) are when manager support will have the biggest impact on learning transfer and use. Review learning and performance objectives to identify critical PFIs, and then build application exercises around them.
- Develop Application Exercises
Once you’ve identified the PFIs, create brief exercises for the learners that are targeted to specific skills, and then require their managers to provide feedback. The key is to create exercises that learners can complete in five to 10 minutes, with manager feedback taking about the same amount of time. For example, a management program may ask the learner to create a meeting agenda to practice meeting management skills. Then, the learner’s manager would receive the agenda and provide feedback.
- Implement Micro-coaching Feedback Loops
One key reason managers give for not coaching is lack of time. With the right feedback loop, application exercises provide the perfect opportunity for managers to spend minimal time providing feedback with maximum impact. The key is to provide managers with tips and reminders on the key criteria they should use to evaluate the exercise. Using the previous example, you could provide managers with a bulleted list of what learners should include in a meeting agenda. They can then reference that list when giving feedback and rating the quality of the exercise response.
- Design Reports to Drive Action
The problem with most evaluation reports is that they come months after the program is complete, are looked at once and then are put on a shelf. This approach misses the window of opportunity to truly affect performance. To avoid this problem, create a dynamic reporting capability, and provide it to managers to show both the performance and the learning progress of their employees.
For example, each week, the manager may receive a message that lists his or her employees and displays their progress on the learning journey (ahead, on track or behind), as well as their average application exercise performance rating (exceeds, meets or fails to meet expectations). Formatting the as a color-coded heat map enables managers to quickly see who is behind and/or fails to meet expectations so they can quickly act.
If you need an executive summary report, you can compile managers’ ratings of assignment performance and their comments into an overall report. This report has a higher degree of credibility with executives, because the data comes directly from the managers. Thus, not only do you provide the learners with valuable feedback, but you also give executives valid and credible performance data. Because the ratings are on the application of learning, the evaluation moves beyond knowledge gain to behavior change and performance metrics.
By centering impact evaluations more fully on managers’ needs, you can better engage your organization in driving performance change and reclaim evaluation as a necessary element of learning.