In the TV series Star Trek, the holodeck on the USS Enterprise spaceship was technologically pure fiction. Like many technologies from science fiction, mixed reality (MR) is conquering more and more areas of our lives. MR integrates digital content into the real world and is becoming an important and versatile key technology of the future in the field of vocational education and training. Let’s look at one case study.

We are in the driver’s cab of the bucket wheel excavator 288 of RWE Power AG. It is 13,500 tons, 220 meters long, 96 meters high. It runs at 10 meters per minute and has a conveying capacity of 240,000 cubic meters of overburden per day. In front of us, we see control elements with which we can put the miracle of engineering into operation. We learn to switch on the main control voltage for the machine, to operate the auxiliary drive bucket wheel, to swivel the bucket wheel boom, to lift the lifting gear bucket wheel boom and to let the machine conveyor run unlocked, without a push belt of course.

In order to learn all this, we no longer have to go to the location in Germany. We don’t even have to climb into a cab. All we have to do is put on a pair of mixed reality glasses and launch an application developed by TÜV Rheinland Academy in cooperation with RWE Power AG, part of the RWE Group and a leading company in Germany for energy production and generation. The application allows untrained maintenance personnel to learn how to operate the machinery with a significantly higher level of safety.

The Challenge and Solution

At RWE Power, technical maintenance and repair work is required at certain intervals on the large-scale equipment. Until now, the control of the excavator during this work was the responsibility of the equipment operator. Moving forward the maintenance personnel will take over this task after successfully completing a training course with MR glasses.

Previously, the training for maintenance personnel to operate large-scale opencast mining equipment at RWE Power was exclusively delivered via on-site instruction. Now, with the mixed reality glasses, the driver’s cab can be brought into the classroom. First-time users can now gain a spatial impression and initial orientation without having to be close to the large-scale equipment.

A pair of mixed reality glasses is required for this training experience. RWE Power AG used the Microsoft HoloLens. It displays holograms, projections from light, in the user’s environment. Sensors in the glasses scan the environment and produce a digital copy. Therefore, holograms can be placed in the real environment using “spatial anchors.” Holographic objects appear like real objects. Models are visible from all sides and retain their position in space as the user moves.

In the glasses’ field of vision, functions of the switching elements are explained in “information mode” and operations in “guide mode” are explained step-by-step. All modes are audio supported. The use of the technology leads to practical experiences in the classroom without exposing the learners to the dangers of operating expensive, unfamiliar machinery. The modes have already been successfully evaluated in initial user tests at RWE Power at the end of 2018.

Another important milestone in the rollout of proof of concept is scheduled for 2019. After the successful user tests, the maintenance application will be transferred from the classroom to the large device. In this phase, the application is no longer executed in the classroom, but within the real excavator.

In the “information mode” and “guided mode,” holograms superimpose the real physical operating elements in the driver’s cab. In this phase, the employee receives performance support during the maintenance work, also called “work support mode.“ All control elements are highlighted for defined workflows. The user can work independently and does not have to interact with the glasses; additional assistance is available as required. These modes are particularly suitable for refresher trainings if the employee has not used the large machine for a long period of time.

Immersion increases motivation and learning success

Perception research has shown that 90 percent of what we actively use and practice is retained, especially when implemented in a virtual or enriched environment where we are supposed to use it. Additionally, technology offers the option to proactively retrieve content in mixed reality training and provides remote support from experts. MR provides learners the advantage of safely training on virtual objects in the classroom. Presence seminars and virtual formats complement each other ideally.

Here are the biggest advantages of mixed reality in the area of education and training at a glance:

  • Beginner friendly: The training solutions can be used directly by all employees and trainers in the workplace or in classrooms.
  • Visual: Animated learning content is displayed holographically and eliminates cost-intensive and time-consuming physical learning objects. Essential features are impressively highlighted in MR.
  • Context-related: Digital content superimposes physical devices, facilitating the development of practical skills in the workplace.
  • Immersive: Complete immersion in the learning situation creates a flow state for optimal readiness to receive the new content.
  • Self-controlled: Interactive content allows hands-on learning at one’s own pace. Playful tasks increase motivation.
  • Social: Learning experiences can be shared across devices, locally or globally, and guided by a trainer for any individual group.
  • Didactic: There is a direct connection between theoretical content and practical applications for the learner.

These arguments quickly convinced RWE Power that MR was the right choice and the best format for its learning objectives and application scenarios, and also promised the greatest learning success. Furthermore, it was important for RWE Power to be able to use only one device worldwide for various training scenarios. With the mixed reality device, the Microsoft HoloLens, this requirement was also fulfilled. The application can be carried out without cabling or complex spatial installation and without a smartphone or additional computer. The user no longer needs to sit firmly in their seat at their desk but can move freely, whether in the workplace or in the classroom.

MR makes new forms of interaction possible. The virtual content learners see can be recorded as a video capture, allowing exercises to be repeated without glasses. Furthermore, learning success can be strengthened through visual memory. For RWE Power AG, the application of mixed reality training is a welcome pilot project to discover the fascinating possibilities and opportunities of this technology. Wilhelm Stock, head of technical training at RWE Power AG, and employees ranging from engineers to large equipment operators are enthusiastic about the technology and are already thinking about further application scenarios: power plant applications, live support and the transfer of best practices in the production process via virtual online simulation.