Ultrasound is the stethoscope of the 21st century. It is the only imaging tool that can be brought directly to the patient rather than forcing the patient to go to the imaging device. Not only is that better for the patient who may not have the mobility but it also allows for imaging scheduling to be more flexible, available and safe during a time where infectious diseases can be transmitted in a waiting room. In fact, for even greater flexibility in acquiring diagnostic images, the probes on the ultrasound device can now be attached to a cell phone.

Supporting High-demand Clinical Skills

Like many universities, Vanderbilt has a growing nursing student population. Between an aging population with a longer life expectancy and advances in healthcare technology, nurse practitioners are in high demand – especially those with ultrasound training. Aside from scheduling convenience and practicality factors, sonography is quickly becoming a staple of modern medicine and is considered the gold standard for monitoring a developing infant’s health throughout pregnancy. As a result, Vanderbilt has a strong interest in training nurse students to become proficient with the technology.

However, the typical training delivery – which relies on face-to-face training with hands-on experience – is just not scalable for the magnitude needed. There are not enough instructors or ultrasound devices to come close to meeting the demand. The cost of hiring additional instructors and purchasing enough devices to provide that level of hands-on training would be hundreds of thousands of dollars, which is far outside Vanderbilt’s budget.

Like many institutions, Vanderbilt is not alone in this. Every nursing program and medical center faces a serious shortage of trainers and devices that can be released for training and removed as a clinical resource.

The Financial Challenges of Scaling Training in a Clinical Environment

Since much of the education that students receive is already online, migrating ultrasound training to an online environment is a natural progression. However, traditional authoring tools have limitations. While eLearning can show students how to set up the ultrasound console, it cannot adequately demonstrate to clinicians how to properly use the probe on a patient, especially when trying to identify the anatomy being scanned. Furthermore, it’s difficult to show the ultrasound output in real-time or recreate the experience without some form of immersive media.

Healthcare educators, both in academic and practice settings, continue to seek the most realistic simulations for their students. However, the rising cost of healthcare equipment – alongside a heightened focus on patient safety – make it difficult to provide ample opportunities for students to practice new skills.

Like most medical devices, ultrasound machines are expensive; each new machine can cost over $100,000. While portable probes that can be attached to a smartphone are much less expensive, they are still very costly. A broken probe can still cost thousands of dollars to replace.

The Benefits of Virtual Reality

Since Vanderbilt had a limited number of devices and instructors, students needed a safe environment to simulate skill development. By replicating the clinical setting in virtual reality (VR), students are more likely to hit the ground running when given hands-on time with a device and a patient. Furthermore, the chances of a student encountering patients with specific and diverse problems during training sessions are low, so VR provides an opportunity to interpret multiple scenarios and patient conditions.

Finally, with a VR application, students can replay the activity until they feel confident in their understanding of the task. They can watch the clinician place the probes on the patient, then go back and see what is being imaged. Moreover, the ultrasound output is annotated with text, arrows and circles corresponding to the presentation of the instructor.

By preloading much of the instruction into an online format, the need for face-to-face instructor time is substantially reduced. This allows more people to be trained while reducing the hands-on time required with the device.

Simulating the Experience

To capture the experience of operating ultrasound equipment, three different media components were combined to simulate the clinical environment: 360-degree video of the clinician and patient interactions shown from two distinct camera angles; a top-down view with graphic, anatomical overlays; and streaming video of the ultrasound output timed to coincide with exactly how the instructor is interacting with the patient.

A 360-degree video camera was placed opposite the clinician at eye level, continuously recording the clinician as he or she interacts with the patient and mirroring the student’s subjective perspective. The output from the ultrasound device was captured on a digital video recorder, and the video was moved to a portable USB drive. Lastly, a ceiling-mounted camera was built into the lab to record the same patient encounter from a top-down perspective. With an overhead view of hand positioning and movement, a representation of the patient’s internal anatomy can be superimposed on the image of the patient, allowing the student to see exactly what is being scanned.

Once ready for editing, the videos were imported into Adobe Premiere. Both the top-down view and the ultrasound output were matched the clinician’s actions and precise timing. Then the videos were parsed into shorter video scenarios and saved as MP4 files in an equirectangular format.

Authoring the Learning Environment

Each MP4 video file – representing a specific patient encounter – was imported into CenarioVR, a VR authoring tool for creating immersive learning courses. Each video scenario was enhanced with additional instructional elements on the top-down view of the patient, such as drawings of anatomical elements that appear when selected. Questions, annotations, buttons and hotspots were included to enforce key concepts and quiz the students on probe placement. Once each scenario was finalized with interactions and assessments, the course was published in two formats: HTML5 for viewing on a computer’s browser, as well as CenarioVR Live and mobile for viewing on a phone in a Google Cardboard or Oculus Go headset.

A Promising Outcome

Despite modifications that could be made to improve the immersive experience for students, the VR course is proving to be effective. Since rolling out the immersive learning module in the spring of 2019, each student reported they had a more efficient training experience with the ultrasound probes. As a result, Vanderbilt will continue to expand ultrasound training to support its growing nursing school enrollment while keeping costs down by making the training more efficient and scalable.