Books and articles written on how to measure the impact of training are abundant. There have been many forums and roundtables with training experts advising and guiding on how to measure and quantify training’s impact to the business. Most, if not all of these information sources target the measuring of training impact at, what could be called, the micro-training impact level. That is, isolated programs or initiatives are selected, specific program goals from the sponsor are gathered, training interventions developed and implemented, and costs and results are measured to demonstrate the impact or return on investment using an experimental design framework.
This is all well and good for these isolated programs, but in many cases, corporations provide investment dollars to a large training organization and this investment is spent across many programs and initiatives, some easily isolated, others not so easily. For example, a company may create and introduce a number of products that require skilled people to market and sell. These same products also require skilled product support representatives to ensure high levels of customer satisfaction and loyalty. The training budget has to be allocated across the full spectrum of sales and support, across a number of product areas and sales and marketing initiatives. At the end of the day the management of a particular business unit may want to know from the training organization, “How did my investment in your organization make an impact on my business?” The broader question is not how training impacted the sales and support of a specific product or initiative, rather how did the investment in training as a whole impact overall business results. As another example, it is common knowledge that training budgets are one of the first budgets to be cut when overall expenses need to be reigned. To better guide investment decisions knowing the quantitative relationship between training investment and a business measure, customer satisfaction score would be needed. This level of training impact is referred to as the macro-training impact level.
Most training experts will tell you the macro-level of training impact is the most difficult or impossible to determine because it is too complex or has too many variables involved. Some experts may even suggest that the way to assess the macro-training impact is to aggregate all the micro-training impacts, that is, set up specific studies for each and every aspect of the business focus to isolate the “marginal effects” of training and add these all up. As a result of the perceived complexity of determining a macro-level view of training impact, there is very little, if any, mention of this level of analysis in the training literature.
Does this mean that training organizations can only provide micro-training impact studies around selected isolated programs as the basis to measure their value to an organization? The answer is a resounding no. A theory of training impact can be used to model the relationships between training investments and business performance. To help represent a theory of training impact, a conceptual framework taken from both sociology and econometrics called path analysis is introduced.
Path analysis, and the resulting path model, is a graphical way of presenting and testing theories as cause and effect relationships among a set of variables. One example of a path model for training impact that theorizes the effect that training investment has on employee competency, morale, performance and customer satisfaction is represented in Figure 1.
Borrowing terminology form econometrics, training investment is referred to as an “exogenous” variable, since its values are determined by factors outside the causal system being modelled. The variables, training quality, training availability, competency, morale, performance and customer satisfaction are referred to as “endogenous variables” since the values are determined within the causal system.
Once a path model is defined, analytical methodologies can be used to estimate the magnitude of the relationships in the model. Once the relationships have been derived and magnitudes determined a concept called the reduced-form model can be used to directly relate endogenous variables to exogenous variables. In this case customer satisfaction, morale, performance, competency, training quality and training availability to training investment, thereby providing a direct measure of training impact as shown in Figure 2.
In a series of articles to follow, the concept of path analysis will be introduced and how it can be used in principle to represent theories of training impact, derive quantitative relationships, and represent reduced-form models. The purpose of the articles will introduce the path analysis concept to the training community in the hope that many will see its utility for studying and modeling training impact at a macro level and foster an untapped area of research and application that will complement the experimental design approaches already widely used.
