In 2011, at the first International Learning Analytics conference, learning analytics was defined as the “measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” Early learning analytics looked more like data mining. Patterns and trends could be identified, but modeling and business intelligence did not enter the picture in any meaningful way.
With the evolution of online learning and the integration of learning management system (LMS) technology, human resources (HR) leaders and learning and development (L&D) managers responsible for employee development and workforce training were able to track, manage and analyze training programs and learners’ actions and progress more efficiently. The focus moved from reporting and archiving the utilization data to analyzing it, aligning results with business goals and measuring organizational impact.
Today’s eLearning technology makes it easy to track the learner’s progress, course completion and quiz scores. These factors can be tied to concrete learning and organizational goals, including HR policy and safety compliance and upskilling. But don’t stop there.
What Key Performance Indicators (KPIs) Should You Absolutely Measure?
Course Status and Completion
Course status and completion measures whether the learner moved all the way through the course or stopped somewhere along the way (and to know, act and report on this in real-time 24/7). These course analytics provide critical insight into employee development, safety training, regulatory compliance, career pathway accomplishments and upskilling interest, as well as the total views per course. It also helps administrators pinpoint areas where comprehension drops or other issues arise, such as cultural interpretation or excessive length of a program given available time to learn. Any solid LMS will include this basic feature, and most allow for customization to fit your company’s needs.
Department reports give successful completion levels for each department. They help you assess performance at the manager, instructor and learner levels, helping identify strengths and weakness of individuals, units, and/or instructors, and alerting you to potential compliance issues. Role and department-based assessments are included in stronger LMS platforms. Learn more about what to look for when buying an LMS.
A course timing report measures how long it takes each learner to complete the course, providing an indication of whether the content may be too difficult or poorly explained or too lengthy to be grasped within the average learner’s attention span (15 to 20 minutes). Almost all LMS platforms have this feature built in, and more advanced LMS platforms provide addition insight into the number of stops/starts to complete a course.
A question timing report measures how long it takes learners to answer specific questions. A question that participants take a long time to answer may indicate a need to rephrase the question, to adjust teaching style or to consider cultural issues. Most LMS platforms track timing-to-complete a quiz. Make sure to ask about features to track individual questions within a quiz and attempts per user.
Discussion Forum Activity
Measuring discussion forum activity, or the number of meaningful interactions for a course or set of courses, tracks knowledge sharing and collaborative learning. At the most basic level, KPIs include participation rates (number of participants/entries), with the more advanced or customized programs scoring quality of content, learner to learner, and learner to instructor engagement. Hard data, like discussion forum or chat board participation tracking, can be integrated into eLearning platforms for an additional fee, particularly those focused on higher education.
Soft data, on the downside, requires an administrator or instructor to read the entries, collecting information on how employees are learning and what parts of the course content and structure are helping with learning and satisfaction.
Learner evaluations measure the degree to which the learner absorbed the information. To ascertain the learner’s proficiency and retention of the content, most learning platforms include assessments (quizzes or exams), final score, metrics for number of answer attempts, total time spent training, personalized learning paths or course assignments, and certification or licensing metrics. Most likely, your LMS platform won’t be feature-rich with built-in learner analytics but will include collective learner assessments and data visualization tools. Ask about integrations that include engagement and behavioral patterns, gap analysis, predictive modeling and integrations with HR information system (HRIS) platforms.
Course evaluations are completed by the learner and measure reaction to the course — a self-assessment of their increase in knowledge, applicability of the knowledge and the effect on the organization (from the Kirkpatrick Model). Having this information will help curriculum designers to continuously improve courses and delivery. Course evaluations can be given in survey form at the end of every course, or periodically/randomly to learners within and external to streaming and LMS platforms.
Why is onboarding here? Well, if you expect your employees to operate equipment safely, use technology effectively and not burn the place down, you want to make sure they are well trained. You need to measure content and knowledge relative to a specific set of skills. An argument can be made that this category includes pre-hire, onboarding, and education and development throughout employment. For onboarding, include content packages for HR, safety training, workplace culture, regulations, and sector specific training, and establish KPI per content set. Authored, curriculum specific or ready-to-go training content can be used. Most streaming and LMS platforms do not come packaged with content, so look for industry leaders who package content and technology.
Let’s Not Forget: Other Employee Analytics to Consider
Bottom line, training and development programs are intended to improve individual performance and business operations. Companies frequently connect development programs to organizational KPIs such as improved customer service ratings, reduced violations, fines/fees and accidents, employee engagement and retention.
Employee engagement directly affects an organization’s return on investment (ROI), as it lowers turnover and increases productivity, innovation and resilience to change. Gallup reports that 36% of U.S. employees are engaged in their work and workplace, and 15% are actively disengaged. In the current environment of remote and overworked white, blue and pink-collar employees, there is a real impact that can be gained from measuring engagement. We recommend reading Gallup’s meta-analysis report to learn about the effect of highly engaged teams on 11 business outcomes, including productivity, turnover and profitability.
According to Gallup, seven in 10 white-collar workers are still in remote work situations in the U.S. For employees and employers, the need for learning and development hasn’t gone away with a virtual workforce. L&D continues, and eLearning — whether using virtual live instructors, asynchronous platforms or both — has again proven its power as we continue to wade through the pandemic.
How is your eLearning program affecting the way employees feel about the organization? Employee engagement surveys and pulse surveys (quick, real-time snapshots on a single topic) can measure how employees are feeling about their development opportunities, the workplace culture and even employer loyalty.
Connecting training and development to performance management (and measurement) can be an informative and helpful process, and it is easy to accomplish using today’s performance management technology, HRIS platforms and API integration software. As performance management processes move toward more frequent or continuous touchpoints in managing employee performance, it makes sense to link continuous learning to continuous evaluation of performance. Several of the LMS metrics outlined in this article give managers critical insight to guide employee goal setting. These insights can be integrated into performance management technology and tools. Managers can also use key metrics to identify employees who are in a position to mentor others who may be struggling.
One of the top reasons employees accept or remain with a job is professional development, and it can be costly (up to nine months of the position’s starting salary) to replace an employee. A eliminates the barriers for employees to learn, allowing individuals to select the most convenient time, location and device. Want to know whether your development and workforce training efforts are paying off in employee retention? A cross-tabulation of metrics (employee tenure and eLearning utilization) can inform employee retention efforts.
Today’s smart eLearning solutions provide your organization with the content and tools to deliver, manage and track employee development and workforce training. Paying attention to learning analytics not only improves workforce training and employee development, but it leads to benefits for employees, employers and the bottom line.