Immersive Virtual Reality Medical Simulation: Autonomous Trauma Training Simulator
Medical and traumatic emergencies can be intimidating and stressful. This is especially true for early-career medical personnel. Training providers to respond effectively to medical emergencies before being confronted with a real scenario is limited by unnatural or high-cost training modalities that fail to realistically replicate the stress and gravity of real-world trauma management. Immersive virtual reality (IVR) may provide a unique training solution.
We created a working group of 10 active duty or former military emergency medicine physicians and two technical experts. We hosted 10 meetings to facilitate the development process. The program was developed with financial support from the Telemedicine and Advanced Technology Research Center (TATRC), through the primary vendor Exonicus, Inc, with support from Anatomy Next Inc, and Kitware, Inc. Development was completed using an agile project management style, which allowed our team to review progress and provide immediate feedback on previous milestones throughout its completion. The working group completed the resulting four simulation scenarios to evaluate perceived realism and training potential. Finally, testing of the technology platform off the network in a deployed role 3 was conducted.
Upon completion, we created four IVR scenarios based on the highest mortality battlefield injuries: hemorrhage, tension pneumothorax, and airway obstruction. The working group unanimously indicated a high level of realism and potential training usefulness. Throughout this process, there have been a number of lessons learned and we present those here to show what we have created as well as provide guidance to others creating IVR training solutions.
Our team developed trauma scenarios that, to our knowledge, are the only IVR trauma scenarios to run autonomously without instructor input. Furthermore, we provide a potential template for the creation of future autonomous IVR training programs. This framework may offer a dynamic starting point as more teams seek to leverage the capabilities IVR offers.
Immersive virtual reality (IVR) can be highly effective as a medical simulation training platform [1-7]. Given recent advancements, this technology has become increasingly portable and visually realistic. While IVR technology appears to hold promise, there is a great deal to learn about the best way to functionally develop, implement, and share these training resources. Several commercial groups have created models that recreate current simulation lab environments with instructor input. While these systems increase training opportunities, decrease equipment needs, and offer broad potential, they still require a skilled trainer to ‘prompt the system’. Removing this limitation seems like a potential way to increase scalability. We are currently in the process of creating, to our knowledge, the only IVR simulator that would offer immediate autonomous feedback to users through both real-time patient physiologic responses and overall grading . We will present phase 1 of this (multiphase) project.
Materials & Methods
We created a working group of 10 active duty or former military emergency medicine physicians, and two technical experts who had programming and IVR project experience. We hosted 10 meetings to facilitate the development process (results). The project was completed at Madigan Army Medical Center in Tacoma, WA. The program was developed with financial support from the Telemedicine and Advanced Technology Research Center (TATRC), through the primary vendor Exonicus, Inc, with support from Anatomy Next Inc, and Kitware, Inc. Development was completed using an agile project management style, which allowed our team to review progress and provide immediate feedback on previous milestones throughout its completion. The working group completed the resulting four simulation scenarios to evaluate perceived realism and training potential. Finally, the technology platform was tested in a live, off the network, deployed environment in Iraq at a role 3 facility (deployed military hospital).
Upon completion of phase 1, we have created four IVR scenarios based on the highest mortality battlefield injuries: hemorrhage, tension pneumothorax, and airway obstruction. Throughout this process, there have been a number of lessons learned. We present those here to show what we have created as well as provide guidance to others creating IVR training solutions (summary shown in Table 1).
Virtual reality platform
We reviewed the technical specifications of the current IVR platforms (Mobile, Microsoft Mixed Reality, HTC Vive), and augmented reality platforms (Hololens). A smaller focus group tried each platform and reviewed computing, graphics, network connectivity, and space requirements. Given its portability, graphics capabilities, and computing potential, we opted for primary development using Microsoft Mixed Reality.
Select a training goal/simulation plan
We initially sought to create a visually realistic training environment around one case. However, this seemed to limit immediate training benefit. Furthermore, the environment may be visually real, but most IVR platforms lack easy and scalable methods to change ambient temperature, moisture, or produce complex haptics. As we learned how to incorporate the physiology engine, we shifted towards a complex medical decision trainer. This could be placed in an endless number of environments and internally scaled to multiple patients in future iterations. The user is presented with an unstable trauma patient with a random injury. The physiology engine settings were selected to result in the patient’s death in 2-2.5 minutes if the player does not identify the injury and complete appropriate intervention. The player must keep the patient alive for a minimum of five minutes. Keeping the patient alive is the primary endpoint for the user. Additional factors, such as whether or not a complete assessment was performed, are tracked/graded, but do not affect the patient’s immediate survival. Standard trauma care actions are also available and tracked.
Selecting the case
Several individual cases, procedures, and environments were considered. However, selecting a single case or procedure seemed to drastically limit the scalability. We selected a generalizable trauma scenario for a few reasons. First, it allows multiple branch points to individual procedures (minisimulations) in future iterations. Second, several open-source physiology engines exist to run the physiology in these cases. The individual scenarios were further built to allow three progressive levels of consciousness if an injury is not addressed in time, and a failure state (death) that can be reversed if an injury is identified and treated.
The room and 3D objects
We sought to include every three-dimensional (3D) object that could be required in caring for a trauma patient, which totaled 36 items. The room was based on a 3D rendition of a standard military trauma bay.
Visual, audio, and exam cues
The selected injury scenario dictated visual/audio/exam cues (as in a traditional simulation).
The incorporation of the physiology engine (Pulse) allowed us to develop a dynamic and more realistic simulation. Tying the simulation timeline to the physiology engine allows a player to see realistic vital sign changes, and complete the course of care as they would in a real patient scenario. It also allows for rapid case variations and randomization.
Team member interaction
Trauma management is a team sport. While a multiplayer/multidisciplinary approach is optimal, this again requires multiple skilled professionals to participate simultaneously. We sought to automate this process through two computer characters: a Nurse and a Medic. Commands are given vocally or through gaze-activated menus.
Master action list/grading scheme/feedback system
A list of all the potential actions a player could make was developed (132), and each was tied to a specific outcome (injury treatment, lab availability, medication administration). These outcomes are injury and level of consciousness specific. Furthermore, each action was tied to a grade based on useful actions, neutral actions, and harmful actions. The Joint Trauma Committee Clinical Practice Guidelines and Advanced Trauma Life Support content were incorporated into the grading schematic.
The working group members have been end users of several computer training solutions, and sought to minimize technical frustration. The primary mode of interaction is using 3D objects to trigger animations. There are also multiple duplicate pathways, for example, starting intravenous access via a 3D object, voice command, or a menu selection.
It was readily apparent a thorough self-guided user tutorial would be necessary. We included key interventions and steps that would allow the user to ‘interact’ in the virtual environment.
Gaze/location tracking system
Given the user is completing actions in a digital world, it is quite easy to track multiple data points that may further relate to performance. We incorporated a gaze/location tracking system to allow for more analytics.
Upon completion of the trauma scenario creation, the working group unanimously indicated a high level of realism and potential training usefulness. The technology platform worked in a deployed environment without internet connectivity (Figure 1), further highlighting the capabilities of this autonomous IVR system for military training.
Our team developed four trauma scenarios that per our knowledge, after an extensive literature review, are the only IVR trauma scenarios to run autonomously without instructor input. Furthermore, we provide a potential template for the creation of future autonomous IVR training programs. This simulator, and IVR broadly, still has several limitations. First, the use of IVR can induce side effects (headaches, dizziness, and nausea). While the advancements in technology have greatly improved this, it is still an issue for some users. Second, it is challenging to create a seamless verbal feedback mechanism. Microsoft Mixed Reality does have voice recognition capabilities, but this produced mixed results when asking the simulated patient questions. Finally, the hardware and software are expensive. They both require initial development and upkeep. However, similar to computers, the cost of IVR systems and programming has been decreasing. Furthermore, these systems are still cheaper and more portable than most mannequin-based simulation systems.
Overall, this pilot project helps reveal the broad potential IVR has for medical training through automated and instructorless scenarios. These and similar self-directed scenarios may create a scalable simulation platform, similar to flight simulation training. We feel this holds immense learning and training potential for medical education. Several challenges still exist including side effects and high costs for development/equipment. Furthermore, more research is needed around learning retention, optimal content, and delivery curriculums. The framework presented in this article may offer a dynamic starting point as more teams seek to leverage IVR for medical education.
- Chakravarthy B, ter Haar E, Bhat SS, McCoy CE, Denmark TK, Lotfipour S: Simulation in medical school education: review for emergency medicine. West J Emerg Med. 2011, 12:461-466. 10.5811%2Fwestjem.2010.10.1909
- Freeman KM, Thompson SF, Allely EB, Sobel AL, Stansfield SA, Pugh WM: A virtual reality patient simulation system for teaching emergency response skills to US Navy medical providers. Prehosp Disaster Med. 2001, 16:3-8. https://doi.org/10.1017/S1049023X00025462
- Hamza-Lup FG, Rolland JP, Hughes C: A distributed augmented reality system for medical training and simulation. Energy, Simulation-Training, Ocean Engineering and Instrumentation. 2018, 4:213-235. Accessed: August 9, 2019: https://arxiv.org/abs/1811.12815v1.
- McGrath JL, Taekman JM, Dev P, et al.: Using virtual reality simulation environments to assess competence for emergency medicine learners. Acad Emerg Med. 2018, 25:186-195. https://doi.org/10.1111/acem.13308
- Price MF, Tortosa DE, Fernandez-Pacheco AN, et al.: Comparative study of a simulated incident with multiple victims and immersive virtual reality. Nurse Educ Today. 2018, 71:48-53. https://doi.org/10.1016/j.nedt.2018.09.006
- Reis S, Guimarães P, Coelho F, Nogueira E, Coelho L: A framework for simulation systems and technologies for medical training. 2018 Global Medical Engineering Physics Exchanges/Pan American Health Care Exchanges (GMEPE/PAHCE). IEEE, Porto, Portugal ; 2018. 1-4. 10.1109/GMEPE-PAHCE.2018.8400757
- Sagalowsky ST, Wynter S-A, Auerbach M, Pusic MV, Kessler DO: Simulation-based procedural skills training in pediatric emergency medicine. Clin Pediatr Emerg Med. 2016, 17:169-178. https://doi.org/10.1016/j.cpem.2016.05.007
- Couperus K: 77 virtual reality trauma simulation: an immersive method to enhance medical personnel training and readiness. Ann Emerg Med. 2019, 74:30. https://doi.org/10.1016/j.annemergmed.2019.08.081