Alisa beamed as she presented the final report for the year-long leadership development initiative she spearheaded at her organization. She recapped the thoughtfully designed, blended training curriculum and its related test scores and comments. She highlighted positive results in key company metrics, including employee retention, customer satisfaction, product sales volume and promotions of the program’s participants.

She eagerly awaited the management team’s reaction. After some pleasant nodding, someone asked, “Alisa, please explain how training specifically influenced the organizational results you have presented, such as customer satisfaction. I would think there are multiple contributors to that outcome.”

Alisa began to sweat; she didn’t really have any supporting data.

If this story sounds familiar, you are in good company. Many training plans, even those for award-winning programs, are missing one critical element that will both ensure their success and demonstrate value to stakeholders: Level 3 data connecting on-the-job performance to program results.

This common omission has cast doubt on the value of training and is a key factor in the cyclical nature of training budgets. However, creating and implementing a plan with the all-important Level 3 data is not as difficult as some assume.

The Basics of Training Value

Begin designing your program using the four levels of training evaluation, or the Kirkpatrick Model. Savvy training professionals know that every training request should first be considered through the lens of what Level 4 result it will support, meaning, what high-level organizational outcome the training should positively influence.

From there, the elusive missing piece should be designed: the Level 3 plan. Both the training professional and the organization should work together to define what needs to change in the on-the-job environment and employee performance to yield the training’s desired results.

Then, the training professional can design the training program (Level 2), and consider the type of environment that would best support the learning effort and eliminate unnecessary distractions (Level 1).

The Missing Piece

Alisa’s final report, like many similar reports, was missing the Level 3 component. She jumped from a detailed explanation of the training program to taking credit for organizational results that, by nature, have many contributing factors. One plus two does not equal four, meaning that formal training alone, no matter how good, will not produce a meaningful level of organizational results and cannot be credited with results without sufficient data showing the connection.

There is strong agreement that formal training alone yields a fairly small portion of any program’s organizational results. On-the-job experiences are the biggest source of learning for employees, according to the “Deconstructing 70-20-10” research report published by Training Industry. An organization’s on-the-job environment and culture significantly impact what employees will or won’t do, regardless of their knowledge.

During the next round of budget cuts training will likely be on the list, because no one can really tell if it was successful or not through anecdotal evidence alone. This is why creating a Level 3 plan and defining roles and responsibilities early in the planning phase is critical not only for the success of the program, but also for the continued existence of the training function.

Creating a Level 3 Plan

The first step in creating a credible Level 3 plan is to define, in observable and measurable terms, the critical behaviors that training graduates should be able to perform as a result of the training.

To ensure training graduates do what they know they are supposed to do, your plan should include a variety of methods and tools to reinforce the importance of critical behaviors, provide coaching and support in performing such behaviors, track the degree to which they are performed, reward those who do them reliably and correct those who do not.

In reality, executives are not physically there on a daily basis, and even direct supervisors often feel that they do not have enough time to support training graduates, or possess high enough comfort level in tracking performance and correcting employees who are not implementing the behaviors taught during the training. Therefore, during the program development process, there must be buy-in for roles and responsibilities after training to ensure the plan is followed. The degree to which the Level 3 plan is implemented defines the initiative’s overall success.

Implementing a Level 3 Plan

Training analytics are often summative in nature, meaning they occur at the completion of a training program or implementation phase. Focus instead on formative evaluation, or that which occurs during a phase, so you can correct issues and maximize performance and results along the way.

Before training, find out what analytics are important to stakeholders, managers of training graduates, training graduates themselves and the training group. Determine who will gather and report the data, and decide how the data should be formatted. Determine which data will be reported throughout the initiative, and how often, to track progress and identify areas for improvement.

Let training participants know before and during training what is expected of them after training, what support is available to them and how their performance and outcomes will be tracked. Making this connection has countless benefits, including increasing training relevance and engagement, because participants can see the training’s purpose and how it connects to their personal performance and contributes to greater organizational success.

When the formal training is over, the important work begins. As training graduates return to the job, schedule regular checkpoints to ensure that the reinforcement, support and accountability actions are occurring. Automated reminders to check in with key parties, such as managers of the graduates, are a good practice.

Expect to have a few setbacks, and be prepared with a remediation plan. This is a normal part of the process, and the reason that continual monitoring and reinforcement is required for success in most programs.

A Leadership Development Example

In Alisa’s leadership development program, a critical behavior for new leaders could be conducting weekly team meetings with all direct reports to review key projects and progress toward milestones. This behavior could be supported by requiring new leaders to submit monthly reports documenting when team meetings were held and the key issues discussed during each meeting. New leaders failing to submit their reports or to hold the required meetings would have a conversation with a senior leader about their reason for doing so.

The underlying belief here is that team meetings foster communication which, in turn, fosters good decision-making that contributes to timely and satisfactory job completion. This will produce pride in the team, leading to employee satisfaction, higher retention rates and possibly awards and recognition showcasing the organization as a great place to work.

These lofty outcomes can have a variety of contributors. This is where the story that underpins the data is required to credibly connect training to performance and to organizational results.

Reporting on Level 3 Analytics

Powerful data is a combination of numbers and information that supports them. This is the formula that connects what is learned to what is done on the job to the results it creates. However, there is always the objection that some other factor created or influenced the results.

In reality, there always remains a number of factors that influence results. With a reasonable investment of resources, you can show the relative contribution of multiple factors, and create a culture of collaborative success.

A simple way to connect training to organizational results is to survey training graduates and their supervisors on what factors have contributed to their success. For example, supervisors could be asked what factors they felt were most important in reducing turnover of their direct reports. They could be force-ranked, or respondents could select all that apply.

Responses could include:

• Skills learned in leadership development training
• Regularly scheduled team meetings
• Supervisors taking a personal interest in their direct reports
• Support from human resources
• Technology and tools available to employees
• Company reward systems
• Office environment
• Company culture/values

Then, respondents can be asked to share a story related to how one or more of these factors contributed to their success. The responses to these questions show the value that training brings to the company overall and highlights other important factors contributing to the company’s success, giving credit where credit is due, and contributing to a team-based approach to success.

If Alisa had asked such a question, she might be able to tell a story like this:

Alan, a new leader, participated in the leadership development program. He learned how to conduct an effective meeting and was asked to hold weekly team meetings. At first, he objected that no one had time to attend and meeting attendance was sporadic. The program he was spearheading experienced a delay, which cost the company money.

With some management pressure, Alan began holding regular meetings. Through team discussions, misunderstandings that were slowing progress were addressed and resolved — and the delayed program was finally launched. Client response was favorable, and a key client increased their purchases by 10% because they were so pleased with the new offering.

Alan treated his team to a pizza lunch on Friday, and presented a variety of awards that were both serious and humorous in nature. Suzanne, a team member, said, “I would never leave a great team like this.”

Taking the First Step

Creating a Level 3 plan can seem daunting for those who have never done it before. Here are a few tips to get started:

• Select one important initiative, and use it as a pilot.
• Find an executive sponsor, and share this article with them.
• Create a committee to support the pilot.
• Spend less time tweaking the training program and more time on what occurs before and after it.

Invest your time in building a process during the pilot. As a result, subsequent programs will be both faster and easier. By following the tips and tricks outlined in this article, you will soon find yourself building and implementing Level 3 plans for all major company initiatives — and reaping the benefits.