In this article, Harry Colangelo, provides a comprehensive guide to calculating return on expectations (ROE) to prove the benefit of employee training to improve performance.

The Purpose of Level 4

The Kirkpatrick New World Evaluation Model is often synonymous with training evaluation. I love the new set up of the Kirkpatrick New World Model visual because it places what is most critical to organizational strength, development, success and/or change at the heart of the model.

(Source: Kirkpatrick Partners)

Wendy Kayser Kirkpatrick and Dr. Jim Kirkpatrick’s graphic model positions Level 3 Behavior in the middle of a visual bullseye connected to the other levels. This connection illustrates that to change business performance so that it aligns with meeting organizational goals, we will need to change our workforce’s behavior.

When we focus on Level 3 Behavior, we can better affect Level 4 results: “desired outcomes” and “lead indicators.” From there, we can make a direct connection from our “desired outcomes” to our “organizational goals/benefits.”

Achieving Kirkpatrick Level 4 Without ROI Calculation

“If you want something you’ve never had, you must be willing to do something you’ve never done.”

In the spirit of this ideal, we must be open to new approaches to measuring the impact of employee training when proving the organizational benefit of our training programs. It’s widely understood that training environments not tied to income-based initiatives but instead, impact on performance, tend to have a difficult time calculating return on investment (ROI).

So, rather than throw our hands in the air, we must consider alternate avenues to demonstrate the value of learning and development (L&D) to the organization and our stakeholders. One solution is to calculate training’s return on expectation (ROE). You can leverage ROE to accurately demonstrate training’s effect on improved job performance to stakeholders.

When we can prove better performance outcomes thanks to training, we can better demonstrate the organizational benefit. Therefore, calculating ROE from our training program is critical. So, how do we calculate ROE for our training programs?

Using Level 3 to get to Level 4.

When calculating ROE, ensure to define a level of success, or in other words, a success metric based on the desired outcome. For example, this can include a 90% knowledge retention rate post training. To measure this calculation, you could include five choices:

Strongly Agree (5) Agree (4), Neutral (3), Disagree (2), and Strongly Disagree (1).

This set of choices is designed with point values, allowing us to calculate an average score that reflects the level of a specific behavior being trained.

With a 90% success metric in mind, we can aim for all behaviors to average at least 4.5 out of 5.0 on our Likert scale. Now that we have a clear standard for measurement, it’s time to establish a baseline.

Establishing a baseline in training measurement.

The baseline represents the current level of behavior that learners exhibit before training begins. It’s essential to use the same measurement tool both before and after training to accurately assess changes in behavior.

A common tool is a Level 3 post-training on-the-job survey given to managers to assess their direct reports. There are many other assessment tools that could work for establishing a baseline, such as a skills analysis or observation. However, in this case, we would ask the managers to complete the survey before training.

About a month after training, we follow up with the manager to complete the same type of survey regarding the job behaviors the employees were trained on. What we now have is a rich data source that demonstrates the affect training had on each of the employee’s job behaviors before and after the training.

We can now calculate and compare it to the organizational goal of 90% success rate in application. Let’s demonstrate in the example below.

In the following steps, ROE is determined using Level 3 data from the training program:

1. Target Value – Pre-Training/Base Value = Expected Difference.
2. Post-Training Value – Pre-Training/Base Value = Actual Difference.
3. (Actual Difference / Expected Difference) x 100 = ROE

Now, consider this example:

The behavior being trained had a pre-training score of 3.9/5.0 on the Level 3 survey, based on a Likert scale average. The desired standard is 4.5/5.0 (90%). After the training, the same Level 3 survey was administered, and the post-training score was 4.6/5.0.

The formula and ROE would then be calculated as follows:

1. 5 (Target Value) – 3.9 (Pre-Training/Base Value) = .6 (Expected Difference)
2. 6 (Post Training Value) – 3.9 (Pre-Training/Base Value) = .7 (Actual Difference
3. (.7/4.6) x 100 = 116% ROE

Wrap Up

We can achieve the elusive Level 4 results to identify the organizational benefit of our training program despite not being able to calculate ROI. The resources are in front of us the whole time.