Simulation has long been used in medical education through cadavers and high-fidelity mannequins to help students practice procedures or interact with the human body in a low-stakes, real-life experience.
With today’s technology, these simulations can now happen in various realities such as augmented reality (AR), virtual reality (VR) or mixed reality.
While AR takes place in the physical world and VR takes place in the digital world, mixed reality is a blend of these two realities integrating real life and the real world with synthetics and computer interaction. At the University of Texas at Dallas, Dr. Marjorie Zielke, professor of research and director of the Center for Simulation and Synthetic Humans, and her colleagues are creating an immersive, mixed reality program for medical students by building virtual humans that students can interact with in a real setting.
Vikram Savkar, Vice President and General Manager, Medicine Markets of Health Learning, Research & Practice at Wolters Kluwer, spoke with Dr. Zielke and her colleague D. J. Zakhidov, a research scientist at the Center for Simulation and Synthetic Humans at UT Dallas, about the future use of mixed reality in medical education in an episode of the Wolters Kluwer Expert Insights Webinar Series.
Key takeaways on mixed reality in medical education:
1. Students need opportunities for social learning with either virtual or real peers
Social interaction is important regardless of the environment. Dr. Zielke learned this firsthand while working remotely during the COVID-19 pandemic, reinforcing the importance of interaction for students to be able to learn from each other. Dr. Zielke’s mixed reality program is built around developing social emotional intelligence. When interacting with the virtual human patient, students also have the choice of interacting with virtual or real professors and peers allowing for collaboration. They can even choose who they want to receive feedback from and who they want to observe allowing them to choose how they want to learn. Another benefit of mixed reality is the opportunity for students to practice communicating with patients, patients’ families and with their peers. The team’s research has found that students like to interact with the virtual patients before trying real-life interactions.
2. Virtual humans can offer students a broad range of experiences – but it will take time
For Dr. Zielke and her colleagues, the goal of mixed reality for medical education is to be as close to reality as possible but natural discussion is innately limited with a virtual human. The virtual human character that the team has created, named Walter, can share very detailed and specific information about his symptoms and conditions, but he cannot carry a natural conversation with a student. The current natural language processing used in Walter allows for the unpredictability of humans but it’s not yet fully realistic. To truly capture human behavior, the characters need to be dynamically generated meaning they are developed by artificial intelligence using data on human behavior, rather than built one at a time based on a single backstory as the team has done for Walter. In this future state, there will be greater opportunity to create diverse virtual characters that help students understand different cultures and lifestyles.
