Data is not one-size-fits-all. It varies based on the audience it’s being served to and the analysis it’s undergoing, and in a business setting, it’s not always available to everyone. It comes in different varieties, and can either be a helpful tool, or an enormous time sink. If used correctly, however, it’s a powerful way to inform strategic decision making.

It seems like data is everywhere these days, and in fact, it is. According to an article in AppDeveloper Magazine, more data will be created in 2017 than all other years in modern history combined. The training industry is no exception. More tools are touting the ability to report on training activity, and xAPI and the Learning Record Store are increasingly popular. With increased availability, it’s important to dive deep into the types of analyses that can be performed on training data to uncover insights that can inform program growth and development. Depending on the analysis, there may be different cadences for measurement and analysis. Regardless, training data should be a routine part of any training professional’s workflow. In this article, the focus will be specifically on customer training and the three distinct buckets or levels of metrics that are important to investigate.

The first type of data to consider is business-level data. This is used to understand the impact of training on the business as a whole, and is typically reviewed by department leaders and executives. Next is course-level data, which is used to gauge the traction of the training program. It uncovers information about the effectiveness of course marketing tactics, and learner retention. Lastly, there’s content-level data, which tends to be used primarily by instructional designers for the purpose of evaluating different content assets. 

A Note about Data 

In companies both large and small, there are challenges around gathering and analyzing data. In small companies, there may not be an existing infrastructure to collect and store the right data. In larger companies, that may exist, but it’s difficult to get the information from different departments, and security restrictions may require cutting through red tape. If data collection is an afterthought, it may be impossible to gather later.

With these challenges in mind, before investing in program enhancements, it’s important to consider if data is available to measure success. Choosing training tools and systems with robust data availability and integrations can help with this by alleviating some of the burden associated with data collection and storage. Additionally, getting acquainted with the business metrics collected at a company and the colleagues associated with managing finance and customer relationship management software will be important for data access down the road. 

Business-Level Data 

ROI, or return on investment, is one of the best ways to understand if business activities are yielding positive outcomes. Business- level data is typically tied to this metric. Why is it so important to measure the ROI of a training program? For one, it helps with resource management and budget allocation. Demonstrating the outcomes of a program can help make sure that the resources allotted to it are maintained, and help with the recruitment of new resources for program growth. It also aids in strategic decision making, by reflecting on real past performance rather than speculation. Keeping these motives in mind while evaluating data can help cater the analyses to different audiences, and maintain focus.

For business-level data in customer training, the audience is most likely training leadership, in addition to executives such as the vice president of customer success or professional services. While this audience cares how much the customers enjoy the learning experience, overall, they’re most interested in the impact of the program on their bottom line.

Surfacing the impact that the program has on key business metrics can help them understand ROI. For customer training, it’s typical to see organizations taking one of three approaches to demonstrating ROI. The approach depends on what is important to a given company and how leadership thinks of training as a lever in the customer journey. For some organizations, training is primarily a lever to improve customer retention, renewals and upsells. Other companies care more about reducing costs, or increasing product usage. All of these outcomes have financial benefits.

When looking at customer renewals, a key question to ask is whether training is a driver for customer health. One way to assess this is by calculating if customers who take training renew at a higher rate than those who don’t. The training course enrollment data and contract renewal data are needed to perform this calculation. Other relevant questions are, given the following accounts that expanded their contract value, what percentage took training? Or, given the following accounts that churned this year, what percentage took training?

While many companies create an ROI story around training revenue, there is also a compelling case to be made around reduced costs. For example, a strong proactive training program will often aid in support ticket deflection. Tracking the number of support tickets per customer or support tickets per topic over time can help demonstrate the impact of a training program that addresses these areas.

Another key area, especially for software companies, is product usage. Increased product usage can lead to increased value for the customer, so it’s important to assess if training helps drive usage. To measure this, capture customer behavior with user behavior software or plug ins. Then, compare time spent using the product with learner session time (time spent in training). Ideally, a well-trained user will spend more time in the product.

Once these analyses are complete, it becomes simple to determine if there is a correlation between training and these positive or negative outcomes. If a high percentage of customers who take training don’t renew, it’s cause to take another look at content and make sure it addresses the right knowledge gaps and value adds. From an executive perspective, failure to address the business need may be acceptable, so long as it’s evident that a data-driven approach is in play to course correct. It’s better to invest in making training meaningful and impactful than to continue to spend valuable resources on a program that’s ineffective. Whichever way things go, when presenting data to an executive audience, it’s useful to elevate the conversation to true business outcomes and align on expectations.

Course-Level Data 

Course-level data, or training engagement metrics are a training manager’s best friend. They help assess if the program is reaching the right people, if the content is engaging, and most of all, the data is often easiest to come by. The audience for this level of data tends to use it to inform their tactics for marketing coursework, onboarding customers and creating new courses.

Since every organization has different needs, it’s important to establish baselines from past performance, or if the program is new, take time to collect initial training data and see how different tactics affect outcomes. The major metrics at play here are course registrations and completions, and student satisfaction.

Registration and completion data provides insight into which topics interest learners, or how discoverable courses are. If a customer registers for a course, but drops out, it’s possible that the training content wasn’t engaging or relevant. Additionally, this information can help optimize for course length, if drop off is consistent on longer courses.

Smile sheets, or student satisfaction surveys, determine whether participants find training favorable, engaging and relevant to their jobs. There are many possible questions to include on these surveys. Using a net promoter score (NPS) question can help align training program reviews with those of the company as a whole. They ask a simple question: How likely are you to recommend this training to a colleague? NPS is globally benchmarked, and many companies and training programs include it on surveys, so it’s useful for comparison.

Content-Level Data 

A training program is of course only as effective as the content within it, and data can help with evaluating content assets, as well as prioritizing content creation and revision. Unfortunately, training professionals often “set and forget” their content. It can be hard to find the time to revisit and rework existing content with the variety of other day-to-day priorities. Since content can become quickly outdated, it’s advantageous to use data-driven approaches to prioritizing content revisions and creation.

The best place to begin is with the low-hanging fruit. Data can be used to determine what content is needed. Start by analyzing the company’s most popular knowledge-based articles, as well as the most popular topics for user education support tickets. Then, create content that addresses those needs. This should prompt quick improvement in those areas.

Next, use data to keep the most popular content fresh. Look at registration and completion rates at the course level to understand where to focus revision efforts. This way, the company is putting its best foot forward for frequently viewed content.

Conclusion

In this world of ever-increasing data, customer training programs can benefit in a multitude of ways through data analysis. From executive proposals for starting new initiatives, to instructional designer content prioritization, data is powerful. A solid understanding of the audience for the data, the specific metrics needed, and the purpose of the analysis will help avoid the risk of being overwhelmed and uncover valuable insights.