No abstract

Introduction: It is now well-accepted that simulation-based learning (SBL) is an important component of medical education. At our institution, we have a state-of-the-art simulation center, and SBL is already incorporated into many medical subspecialties. However, commercially available patient simulators have static faces and lack the realistic depiction of non-verbal facial cues important for rapid diagnosis of neurological emergencies such as stroke. This multidisciplinary project addresses the urgent need for expressive patient simulators by developing acute stroke avatars for use in simulated healthcare training. Methods: Using a previously published and validated method, we developed techniques to display stroke pathologies on a virtual patient simulator (VPS). After obtaining patient or surrogate consent, we obtained source videos from patients admitted to an academic medical center who had experienced acute ischemic stroke resulting in neurological findings such as facial droop, eyelid apraxia, dysarthria, and coma. We then extracted facial features using shape-based modeling techniques, leaving anonymous feature points. Next, we applied a novel algorithm to use these feature points to build accurate computational models (masks) representing the facial characteristics of stroke. We then overlaid these prebuilt masks onto a live facial video stream to generate asymmetric expressions on a virtual avatar. This project was IRB approved. Results: More than 21 videos of stroke patients were made. Once the feature points were extracted from these videos, we were able to develop VPSs capable of expressing realistic asymmetric facial expressions. These avatars were then validated amongst neurologists with clinical experience in diagnosing acute ischemic stroke. Conclusions: This multidisciplinary effort using patient-inspired facial expressions resulted in a tool that aids the stroke education community by making virtual and robotic simulators of acute stroke more varied, interactive, and realistic.

Clinical educators have used robotic and virtual patient simulator systems (RPS) for dozens of years, to help clinical learners (CL) gain key skills to help avoid future patient harm. These systems can simulate human physiological traits; however, they have static faces and lack the realistic depiction of facial cues, which limits CL engagement and immersion. In this article, we provide a detailed review of existing systems in use, as well as describe the possibilities for new technologies from the human–robot interaction and intelligent virtual agents communities to push forward the state of the art. We also discuss our own work in this area, including new approaches for facial recognition and synthesis on RPS systems, including the ability to realistically display patient facial cues such as pain and stroke. Finally, we discuss future research directions for the field.