Professional survival depends on education in today’s world of rapidly accelerating transformation. At the same time, for organizations to stay competitive, they must have a way to measure the success of that education.

Instructional training is typically delivered as a transactional service, limited to occasions when individuals need to acquire new skills. But the pace of change today, especially in technical disciplines, has become so rapid that employees now need continuous education. This need for ongoing education has driven the growth of subscription-based training.

Three educational components based on learner needs

The first component is technology adoption, on which a company decides to deploy some exciting new technology. The IT team must have the right skills before they can start. So, they enroll in a class to learn new skills. When the skills are acquired, the need for training is fulfilled.

The second component involves ongoing management. For centuries, employees learned trades through apprenticeships. When their training was complete, they had all the knowledge needed to practice their trade for the rest of their professional life.

However, the rapid pace of technological change renders this system obsolete. Software and hardware are updated, security patches and protocols are implemented, and systems are expanded and integrated. It’s easy to imagine an IT professional needing to update their skills with information that has changed since their initial training. In the midst of doing their job, they need an occasional lifeline to help them stay productive.

The third component represents another product of our rapidly changing work environment: career development. To stay competitive, companies need employees’ skills to be up to date. Individuals work to keep their skills current to reduce career risk. Others see new skill acquisition as a path to career advancement.

The three components align with the different models for education delivery. For the most part, technology adoption aligns with the transactional model. A defined set of skills is learned in order to complete a project. The other two components of education, ongoing management and career development, require continuous learning, with constant access to skills, information and knowledge. By nature, these are better fitted to a subscription model.

Measuring success of education subscriptions

Educational theorists have suggested that training can be evaluated on five levels:

  • Reaction: how learners feel about the learning experience or training.
  • Learning: how the learner’s knowledge has increased after training.
  • Application: how much the learner is applying what they learned on the job.
  • Outcomes: the effect of the learner’s new skills on the business environment.
  • ROI: the financial return on the training investment.

The easiest level to measure is surely the first one, reaction. It’s simple to ask, “How effective did you think this training was? How useful was the course material?” This provides a reasonably useful yardstick on training effectiveness. The other four levels are more difficult to measure. It’s hard to follow up with a learner six months later to measure real-world training effects – especially in a subscription model where the timing of training may be different for each individual.

Measuring outcomes is also useful. One outcome to measure in IT training is increased employee productivity. The second is improved uptime. It makes sense that there should be a correlation between training and productivity. Increased network uptime might seem like a simple measure, but it’s not always easy to establish causality. If there is an improvement in uptime, was it because of training? Or was an expert hired? Or was it because a vendor fixed a critical bug?

However, it is likely that the best method for measuring educational success is to return to the three components of education and evaluate training with the needs of the learner and of the company in mind. For technology adoption, two measurement approaches stand out. The first is time to value. If a trained team can complete a project in four months, while an untrained team takes six to eight, this is a clear demonstration of value. Another approach is to compare error rates on new projects between trained and untrained individuals. What does it cost to find errors and fix them later? How much money can the company save by reducing errors? This may be difficult to measure with precision, but no one would argue that training doesn’t reduce mistakes.

Looking at the needs of individuals

As for the ongoing management and career development components, it’s important to consider the specific needs of the individual receiving the training. Subscription-based education is typically much more focused on the individual than instructional learning. Individuals frequently make their own decisions about what they want to learn. Often, they proceed at their own pace. At the end, only the learner knows whether the training has met their need.

The individual must be the primary focus of effective evaluation of subscription-based learning. It must answer the questions: What were the individual’s initial goals and expectations? Has the training helped them meet these goals? What elements of the training were most effective and least effective in meeting these goals? Ultimately, how has the training helped the individual on their journey to success?

What makes subscription-based learning so timely and valuable is that it provides learning content tailored to the individual learner’s needs in an on-demand format. Consequently, measuring this type of training’s effectiveness cannot be reduced to a one-size-fits-all assessment. Instead, it comes down to gauging effectiveness at the individual level.