Two months following a great experiential sales training event, the learning leader bragged about how well the trained salespeople were doing. Last month, they sold 83 widgets on average, while their untrained colleagues sold only 60. When he presented the results to the sales leadership team, his enthusiasm quickly waned. They explained that they intentionally invited higher performers to the event as a form of recognition for their performance.
Was the training really a success? Was the sales result fact or fiction?
This learning leader had fallen into a common trap when analyzing data: He didn’t dig deep enough. His investigation came up short, and he failed to make a credible argument that training contributed to the increase in sales.
Many factors can contribute to employee performance, and where possible, it’s important to consider these elements in the measurement plan. A good analysis takes these other factors into account:
- Selection bias
- Demographic factors
- Organizational environment
- Time-related events
Paying attention to these other factors will enhance the quality and usefulness of any analysis. Each factor is described below.
Selection Bias
In the case of the experiential sales training event, by design, many attendees were already high performers. In other words, there was a bias regarding who could attend the training. A best practice for analyzing performance is to compare prior performance to post-training performance for each employee (trained and untrained). Consider the two scenarios below.
In Scenario A, the trainees were already high performers. While their sales did improve, the lower-performing, untrained employees improved more. In Scenario B, the trainees were low performers and realized a dramatic increase in sales. Including prior performance in an analysis adds insight and credibility to the findings.
Demographic Factors
In the context of training, demographic data includes any individual or group characteristic. Examples of individual demographics include age and tenure, and examples of group demographics include department and region.
Consider the following example, in which an organization implemented several measures to improve employee engagement. Did it work?
Adding both individual (age range) and group (department) demographics provides an interesting view of engagement. Engagement increased among younger employees but decreased among older ones. It also increased in the customer service department but decreased in sales.
Now ask, yourself again, did the employee engagement initiative work? Segmenting data by demographic factors can reveal interesting insights that drive further investigation.
Organizational Environment
Sometimes, factors that are inherent in an organization, such as the multi-year implementation of a new enterprise-wide system, levels of trust or the level of manager support, affect performance. In this next example, the manufacturing team requested that the learning department build training around quality to address a high error rate in Division B. An upfront analysis revealed that, while Division B had greater error rates, the problem may have more to do with the quality of the manager. A study of highly supportive managers might provide insights into a better solution.
Time-related Events
Sometimes, broad organizational events can impact performance and need to be accounted for in an analysis. For example, turnover tends to increase after the organization pays bonuses, and new product launches, work stoppages due to supply chain issues and seasonality can affect performance.
In the following example, a company introduced new sales training and saw increases in performance for two quarters and then a disappointing drop in sales. Did the training wear off?
It turned out that the overall industry experienced a reduced demand. By understanding this trend and plotting sales for trained and untrained salespeople, it became clear that training may have helped mitigate the downturn.
(When interpreting data such as the data presented in these examples, remember not to confuse correlation with causation. The causal argument requires more than just a graph!)
The Goal: Deeper Investigation
The preceding examples illustrate how digging deeper into data yields greater insights into the relationship between training and performance. Adding these other factors to an analysis may trigger additional questions that prompt further investigation in order to develop a broader picture of workplace performance. This deeper analysis builds credibility in the interpretation of the data, enabling learning leaders to deliver results based on fact, not fiction.
