Many business leaders want to know how effective their training programs are. As a result, most learning organizations track course participation and performance. But how can you collect the right data to show improvement, and how can you recognize when these results are meaningful? Here’s how to get answers from your company’s data, even with next to no knowledge of technical analysis.
Ask the Right Questions
Determining the right questions helps you establish an approach to your project. Taking the time to think about the individual issues that arise from the project brief will that ensure you obtain the most relevant data. Many project questions are broad and need further dissection. For example, the question, “How useful is the professional Excel training for our employees?” comprises many potential sub-questions, including:
- How quickly are employees able to complete the course?
- How fast and accurate are they in their Excel work as a result?
- How many employees didn’t learn from the class or dropped out of the class?
These questions refer to quantifiable units of information that you can collect: completion date, Excel work speed, Excel work accuracy, failure rate and dropout rate. Data from these sub-questions support the general question by providing units of information relevant to it.
Gather the Right Data
Once you have identified these questions, devise a method to gather data for them. To continue with the Excel example, take a register of participants, log their attendance at each session and gather records of their assessment scores.
You may have access to data from past iterations of the course. Collecting these data can offer a useful baseline to analyze progress. Are there evaluation reports, registers or test results from previous sessions? The more parameters and, therefore, data that are available, the easier it is to identify troublesome areas. In a sample size of 15 participants, one poorly performing individual may make a massive difference to the data set. This anomaly decreases the more data you collect. The key, then, is to take as many measurements as you can, as often as you can. The more data you have, the more accurate your results will be.
Visualize the Data
There are plenty of visualization programs that help you to read data and formulate them into easy-to-read tables, charts and graphs. This step can be a basic way to analyze and identify trends without bringing in a professional data scientist.
Don’t worry if this process does not result in anything meaningful. Trendless data sets are likely due to an insufficient amount of data and the complexity of the problem. Most data-driven projects require several studies over a period of time to obtain a statistically relevant sample. If you do find something of note early in the process, keep it in mind when you carry out your next test.
Don’t Jump to Conclusions
There is no single way to know when the results are meaningful, which tends to be specific to the industry and project. While a 0.01 percent change could be crucial to the pharmacological industry, the same percentage of change in the sales industry might be considered negligible.
Understanding when the results are meaningful, therefore, requires practice and time. Don’t be tempted to make a rash judgment, even if the data appear to suggest progress or deterioration after the first assessment. Negative change, for example, may not necessarily be the result of poor training. The sample group might not be representative, or participants’ expectations could be too high. If you notice a change, step back, make a note of it and keep an objective mindset. Repeat your investigations, and try to alter the parameters.
For your next test, add or change just one thing so you can see how vital those data are to the study. In addition to the time it took for participants to complete the training, how long did they need to pass each of its assessments? How many mistakes did they make in each particular session? For longer courses, what was the drop-out rate, and when did learners start to leave the program? How do participants perform when they are retested six months after the course?
Data is a game-changer. It offers objective results that can immediately show a company where its strengths and weaknesses lie. To obtain the best outcomes, however, you need to do a bit of legwork. Ask questions that generate quantitative, easily tracked information; spend some time collecting data; and don’t be tempted to leap to solutions before you’ve acquired a large enough sample. Practice is all it takes to implement a data-driven approach to your company’s learning and development practices and to ensure that you and your data are speaking the same language.