The impacts of big data analysis can be seen all around us. If you’ve ever bought a product that Amazon’s recommended for you, or found Google predicting the exact term you were about to search for, that’s it in action. So, what makes a big data approach to learning measurement so different from the old models, and why should you use it to measure your learning?
Big Data vs. Traditional Approaches
In a basic sense, measuring learning using a big data approach isn’t too dissimilar from utilizing approaches like the long-established Kirkpatrick, Phillips or Kaufman’s models. When using these approaches, you start by generating a hypothesis that a change you are going to make to your workforce’s learning will affect your organization’s performance. You then measure a baseline, make the change and measure again to see how your baseline data has changed.
The difference with the big data approach is that you start by harvesting and storing data and then look for patterns, often without a specific question in mind; although, you should be aware of the broad drivers for measuring, such as a desire to monitor and improve aspects of a course, or to better understand impact. Then you look deeper into those patterns and analyze the data, looking for correlations which may prompt unexpected insights or results, which you can communicate or use to optimize and improve your learning.
You can utilize big data analysis in a much more in-depth way than traditional methods. Traditional approaches can only look at the impact of your learning on one or two real-world metrics, whereas big data analytics allow you to look for the unexpected impacts of your learning. You could, for example, measure if your learning intervention has affected both your sales figures and your NPS scores, but also if call center staff are using more positive language or providing better descriptions of products. You can then link these to improved performance and business impact.
Getting Started with Big Data Analytics
To start, you need some form of data analysis tool. Big data analytics cannot and should not be performed with Excel. There are some excellent learning-focused analytics tools out there. Find one that allows you to import and utilize your comparative data, rather than focusing solely on the analysis of the learning data.
Once you have this tool in place, you need to get access to your data and get it into your tool. xAPI has given us a wonderful tool for getting loads of data out of our learning interventions, from the standard tracking data like time spent and completion score to more esoteric data points like CPR dummy statistics, and data mined from transcripts of one-to-one sessions. True data analytics, however, also need the comparative real-world performance data (sales figures, NPS scores, call satisfaction scores, 360-degree review data), which can be harder to get.
Most organizations are beginning to utilize this data for other analytics, so it is often easier than you think to get ahold of this and plumb it into your analytics tool. Ideally, you’d collect this comparative data as widely as possible across your workforce for as long as possible.
While a high-quality learning analytics tool will give you an impressive amount of well-structured analytics to get started with, it’s critical to make sure that these are set up correctly and you are analyzing the results properly, as data can easily be misinterpreted. To ensure this is the case, I’d suggest you get help (unless you’re a statistics expert already). Most organizations nowadays have internal data analytics teams who can help, and your analytics tool provider should have experts to help you along.
In the right hands, you’ll be able to correctly assess the validity of data, as well as key elements such as genuine signs of progress. In one case, when the data analytics experts at Watershed helped to create the VISA University digital learning ecosystem, they also helped the organization to evaluate which key learning moments contributed to exceptional leadership development.
There are lots of reasons why any conclusions need to be drawn carefully. Bias, a lack of control studies and variables like employees’ personal learning are a few of the factors that can affect the results of your training, or at least make it more difficult to work out what’s really going on.
Using Data to Predict Behaviors
The level of analytics provided by big data techniques are a lot more detailed than what can be achieved with traditional models. We are finding that utilizing these approaches puts us on the path of what are known as predictive and prescriptive analytics. This is the kind of information that helps the likes of Amazon and Apple’s Siri to be so pioneering and effective, and is considered by many to be the holy grail of analytics.
When your phone tells you how long your journey to work will take, it uses data on the distance between your home and your office, as well as how long the journey has previously taken you. This is a fine example of predictive technology, and it’s taken a step further by prescriptive analytics, which will increasingly allow machines to automatically optimize what happens in the future.
Prescriptive data allows you to notice that a person is showing the same patterns of results that previous learners in the organization have shown. By using prescriptive analytical techniques on the data, you can begin to predict a certain set of results from people displaying the same behaviors, like a computer anticipating moves in a game of chess.
You might observe a particular series of behaviors which have typically led to employees leaving the company six months later, or spot signs which have previously led to people causing a reportable event. Then, in the same way Amazon might take data from a user’s shopping habits, you can see what interventions have stopped this outcome in the past, and suggest (or force!) the user complete these.
Putting Data Analysis into Practice
This is really useful for L&D departments in terms of planning remedial action. A machine might show that a person took certain modules on a training course to improve their knowledge and skills, or learning managers could ask what, empirically, stops certain unwanted outcomes from happening. You can also look at other valuable insights, such as how a learner preferred to learn; if that turns out to be video, for example, you might serve more learning through short films or animations.
The new methods of measuring learning are much subtler and rewarding. For AT&T, which provided focused learning to 243,000 employees with the help of training data, Watershed’s design saved hundreds of thousands of hours of employee and course production time, increasing the time engaged with learning by 25 percent.
Remember that you don’t need to measure anything specific when you set out. The data might initially look unrelated because the patterns and inferences offer an array of correlations rather than more limited data from a single experiment. Over time, this data will go from useful to invaluable, and you’ll be able to truly measure the impact of training.