Virtual Reality Comes of Age - Medical Training Magazine

Virtual Reality Comes of Age

As we see improvements in technology performance and assessment there is also the opportunity to significantly reduce cost of medical education and training. Paul Pribaz reports.

Natalia Grudzien, a second-year medical student at the University of Illinois College of Medicine at Peoria (UICOMP), lifts two controllers up to her waist, squeezes the triggers, and extends her arms to her sides. “This is like a 360-degree computer screen,” she says, tilting her head and lifting her chin. The large head-mounted display covering the top half of her face doesn’t seem to faze her at all. Natalia is clearly immersed in the virtual reality experience; she pauses the lecture with a tap of her thumb and reaches into her pocket to mute her buzzing cell phone. Another tap and she is again staring ahead intently, slowly craning her neck and moving sideways, as if trying to get a better view. “I’ve never been in virtual reality before. I am completely blown away.” Natalia is watching a pre-recorded lecture about the soft tissues of the knee. She’s able to pause, rewind, and, by using a more exaggerated version of an iPad gesture, she’s able to manipulate the digital images projected in front of her eyes. “This is amazing for understanding the spatial relationship between the different structures. I can literally go into the anatomy to see how it all fits together.”

Instructors are enthusiastic about the teaching opportunities in VR.
Instructors are enthusiastic about the teaching opportunities in VR.
All images: Jump Simulation at OSF Health.

Natalia is using a commercially available headset, in this case the HTC Vive, in combination with custom software that has been developed at the University of Illinois and OSF HealthCare. The software allows access to pre-recorded content in a highly immersive format. An assessment module at the end of a lecture lets her confirm that she understands the content. “I really think this is going to be amazing for medical education. You can’t spend hours and hours in there, but versus a textbook it’s much more engaging.”

The HTC Vive head mounted display.
The HTC Vive head mounted display.

Interaction

The evolution of the custom software followed a common constraint in simulation and clinical education, namely, the difficulty of quantifying the value of expensive tools. For Dr. Matthew Bramlet, Director of the Advanced Imaging and Modeling program at Jump Simulation at OSF HealthCare, the lightbulb moment came when he was watching a surgeon interact with a virtual model of a heart. “We had previously printed in 3D our complex cases for surgical planning. While we have been extremely fortunate to cover the expenses of that with a generous philanthropic gift, our goal has been to try and create a sustainable way for other centers to participate in this work. Once we dropped the digital file into a visual 3D environment and let the surgeons play with it, they started telling us what they saw and what they actually planned to do for the case. I realized that if we could capture what he was saying at that moment, we had educational gold and, more importantly, a sustainable path.”

Dr. Bramlet subsequently founded a company, called Enduvo, which has taken the software license and is working to commercialize it.

“Our first deployment outside our own institution is at the NIH’s National Institute of Allergy and Infectious Diseases, where they are using the software platform to train biovisualization scientists in Africa about the structure of tiny pathogens, which they say is helping them design better studies to understand the etiology of disease. They’ve told us that they love not having to stay up so late at night to talk to another continent over Skype – they just record what they want to say with the 3D image and email the entire file.”

Dr. Bramlet cautions that while the software platform makes it easy to teach in a VR environment, there is a significant amount of work required to create anatomically accurate 3D models. “There is definitely some skill and practice required to turn a DICOM dataset into an accurate 3D model. But once that’s done, you can load it and any other digital files, like an image or a movie, into the environment, hit record, and start teaching.”

Not all users of Enduvo are medical students or PhD scientists. Dr. David “Trey” Jantzen, a pediatric cardiologist at the Children’s Hospital of Illinois, has developed a fetal cardiac screening course for sonographers. One challenge in training allied health professionals is that their educational and clinical exposure may vary widely depending on where they trained and where they now work.

“This tool easily helped me to standardize my educational content. Now that I’ve developed exactly what I want to teach, I can more efficiently deliver this content to a broad audience in an immersive virtual environment. I think this saves time for the educator and the learner and should be a lot more fun for the learner as well.”

Dr. Jantzen is conducting a research study to compare effectiveness, cost, and time spent with traditional versus virtual reality education formats.

“It makes intuitive sense that if you don’t need the instructor, you are going to save money. We’re also looking at the learner time spent in the course, since these are practicing professionals who are paid when they go to class. We hypothesize that their actual ‘in-seat’ time is going to be reduced significantly, which will provide an ongoing return.”

A DICOM image must be segmented at each layer to create an accurate 3D digital model.
A DICOM image must be segmented at each layer to create an accurate 3D digital model.

Improved Training

Dr. Teresa Riech, an emergency medicine physician at OSF HealthCare, is a former combat medic, Flight Surgeon, and Chief of Clinical Services for the 183rd Fighter Wing, who has witnessed first-hand the need for improved training in the principles of triage and field medicine for medical officers and enlisted medics.

“We know that training in mass casualty triage helps, but we don’t know what method works best or how realistic the scenarios need to be,” Riech says. (Bhalla, 2015) Riech has planned and executed many large-scale Mass Casualty Incident trainings, but notes that no training exercise simulates the stress experienced in the real world.

“Currently we spend a lot of resources on large-scale live simulations, but we don’t know how well this kind of training translates to performance in a real situation.” (Kilner, 2011) She has created a triage training course in virtual reality, and has started deploying it for pre-hospital responders at all levels.

“I believe this kind of training is very generalizable: hospitals, municipalities, and military operations, from novice to expert learners, would all benefit from exposure to this tool.”

One of Dr. Riech’s primary motivators is the significant time savings that a VR course offers versus traditional training modalities.

“Planning this course took a fraction of the time compared to a live mass casualty exercise. And now I can run an infinite number of learners through it, without the normal expenses of travel, actors, and moulage. Once you have collected all your digital files and imported them into the virtual world, the actual teaching process is incredibly easy.”

Dr. Riech is excited about other opportunities for training in VR. Pre-hospital response for chemical, biological, radiological, nuclear and explosive (CBRNE) situations, or pediatric mass casualty scenarios, are two areas where there is very little in the literature about the best ways to train. “As you can imagine, the stress of responding to pediatric emergencies is much higher, and the requirements for care are more nuanced than adults.”

David Dominguese, PhD, is a Research Assistant Professor of Anatomy and Director of Technology-Based Medical Education at the University of Illinois College of Medicine at Peoria. Earlier this year, he participated in the school’s renovation of its anatomy lab, which included construction of a new classroom for technology.

“A difficult decision to make was on the type of hardware to use for virtual reality, because we know that it’s only going to improve over time. Having access to the Enduvo software platform has made it very easy for me and other instructors, without any software developer support, to quickly build interactive content.”

Professor Dominguese has published preliminary results and presented early findings that both faculty and students’ have very positive reactions to using VR (Dominguese, 2018). His current research is evaluating the effectiveness of VR on learning performance versus traditional methods, and he hopes to publish his findings next year. “We decided that we wanted to investigate teaching and learning using VR and implement into the curriculum early, so that we could pioneer technology and add to scholarship in the field that is vastly needed.”

In the meantime, he and his colleagues are working hard to develop more content that can augment or even replace some of their existing lectures. “We are implementing an enhanced flipped classroom model, where students often spend approximately 30 to 60 minutes in the lab during a schedule week with each block of content. Currently, the biggest challenge is logistical tracking each user’s time spent in VR and what is the most applicable content to deliver in VR.”

Even with these challenges, he and other instructors are enthusiastic about the teaching opportunities in VR. “What we know is that best practices for actual learning are supported with this tool. The learner has the freedom to interact with the anatomy in the most immersive way possible. For me as an instructor, I can receive instant feedback whether or not they are learning.”

Pre-surgical review of patient specific anatomy.

Challenges

As more institutions evaluate the educational promise of augmented and virtual reality, they will have to grapple with the challenge of understanding the rapidly evolving state of the hardware technology, the growing number of software and content options, as well as the physical space requirements for providing a lab. For UICOMP and Jump Simulation, the capital expenses for renovating space and purchasing equipment for a six-person VR lab were significantly lower than the cost of adding additional high-fidelity simulation rooms. Yet even with access to the Enduvo platform, there is still a significant challenge in coaxing educators to try moving their content into VR.

Dr. Bramlet observes that “the biggest energy gradient right now for our faculty is that they need to think differently about how they present their material in VR instead of PowerPoint. Even if I can show them that their one-hour lecture can be reduced to ten or fifteen minutes in VR, they still need to get comfortable with the environment itself and also think about how they are going to present their content and design mini-assessments for each of their learning objectives.”

As arguments that VR is just another passing fad are slowly losing steam with the many improvements in hardware performance and user experience, it remains to be seen whether or not learners and institutions will embrace the financial and intellectual leap of faith required by early adopters. Many believe that there is an entirely new VR ecosystem emerging just as with mobile device app development a decade ago, and Goldman Sachs estimates that the potential market size in healthcare and education for VR and AR software will be $5.8B by 2025. (Bellini, 2016) But VR developers are in high demand and their skills are not easy to acquire. In addition to a traditional software engineering background, they need to be fluent in 3D tools, like Unity and Unreal, platform specific software development kits (SDKs), user interface and user experience (UI/UX) principles, as well as sound design.

In the meantime, Dr. Bramlet is excited about new collaborative opportunities that his Enduvo software platform can provide. “We’re finding that many institutions are much more willing to share curricular content than they were before. Our early customers are honestly just as excited about a marketplace for content as they are about the education and research that they can develop by themselves in VR.”

Mass casualty triage training in VR.
Mass casualty triage training in VR.

About the Author

Paul Pribaz, MS, serves as the Chair of the Technology Committee for the Society for Simulation in Healthcare and teaches a course on Simulation Program Administration at Drexel University’s Master of Science in Medical and Healthcare Simulation (MSMS) program. Paul was the Vice President of Simulation Administration for Jump Simulation, responsible for financial and operational oversight of simulation-based activities in research and education. He previously worked as the Executive Director of SIMnext and of Jump Innovation and facilitated the collaboration of clinicians and engineers to develop next generation educational technologies.

REFERENCES

  1. Bellini H, Chen W, Sugiyama M, Shin M, Alam S, Takayama D. Virtual & Augmented Reality: Understanding the Race for the Next Computing Platform. The Goldman Sachs Group, Inc. 2016.
  2. Bhalla MC, Frey J, Rider C, Nord M, Hegerhorst M. Simple Triage Algorithm and Rapid Treatment and Sort, Assess, Lifesaving, Interventions, Treatment, and Transportation mass casualty triage methods for sensitivity, specificity, and predictive values. Am J Emerg Med. 2015;33(11):1687-91.
  3. Dominguese D, Pan S-C, Harris J, Bramlet MA. Virtual reality teaching and learning platform for medical school education. In: Langran E, Borup J, editors. Society for IT & Teacher Education International Conference 2018; Washington, D.C., United States: Association for the Adv. of Computing in Education (AACE); 2018. p. 1109-12.
  4. Kilner T, Brace SJ, Cooke MW, Stallard N, Bleetman A, Perkins GD. In “Big Bang” Major Incidents do Triage Tools Accurately Predict Clinical Priority? A Systematic Review. Injury. 2011;42(5):460-8.

Originally published in Issue 3, 2018 of MTM Magazine.