How valid are commercially available medical simulators?

Abstract

BackgroundSince simulators offer important advantages, they are increasingly used in medical education and medical skills training that require physical actions. A wide variety of simulators have become commercially available. It is of high importance that evidence is provided that training on these simulators can actually improve clinical performance on live patients. Therefore, the aim of this review is to determine the availability of different types of simulators and the evidence of their validation, to offer insight regarding which simulators are suitable to use in the clinical setting as a training modality.SummaryFour hundred and thirty-three commercially available simulators were found, from which 405 (94%) were physical models. One hundred and thirty validation studies evaluated 35 (8%) commercially available medical simulators for levels of validity ranging from face to predictive validity. Solely simulators that are used for surgical skills training were validated for the highest validity level (predictive validity). Twenty-four (37%) simulators that give objective feedback had been validated. Studies that tested more powerful levels of validity (concurrent and predictive validity) were methodologically stronger than studies that tested more elementary levels of validity (face, content, and construct validity).ConclusionNinety-three point five percent of the commercially available simulators are not known to be tested for validity. Although the importance of (a high level of) validation depends on the difficulty level of skills training and possible consequences when skills are insufficient, it is advisable for medical professionals, trainees, medical educators, and companies who manufacture medical simulators to critically judge the available medical simulators for proper validation. This way adequate, safe, and affordable medical psychomotor skills training can be achieved.

Towards Expert-Based Speed–Precision Control in Early Simulator Training for Novice Surgeons

“…If this occurs, especially early in a training sequence, there is either a problem with the simulator task (i.e., the task does not produce adequate performance data that allow discriminating between levels of expertise, which is a problem that needs to be fixed), or the trainee is not a true novice. No more than four cases (1)(2)(3)(4) need to be considered for generating clear performance feedback any novice can understand and take into account to attain an optimal speed-precision trade-off.…”

“…Finally, whatever the simulator, a single performance metric inevitably gives a partial assessment of user performance [20]. Task completion time as a sole criterion has been explicitly demonstrated to be a poor or even misleading measure of surgical skill [1,23]. Some metrics assume a simple global optimum value, such as a minimal tool path length, or a minimal completion time, and other quantities such as forces [6,21] or velocities [23,[33][34][35].…”

“…The complexity and reliability of commercially available simulators varies considerably, and selecting an appropriate simulator for surgical skill training is in itself a challenge. Simulators for specific surgical skills are generally tested for the highest validity level [1], that of predictive validity, ensuring that assessments of performance in the specific simulator task are likely to predict future performance of the trainee in an equivalent task in the clinical context (animal, patient). Only a certain percentage of surgical simulators provide some kind of performance feedback to the trainee.…”

Achieving interface and environment fidelity in the Virtual Basic Laparoscopic Surgical Trainer

“…In addition, a recent study has shown that even when these systems are available at the teaching hospitals, they are underused because they are physically locked away, perhaps owing to their cost (Brennan, Loan, Hughes, Hennessey, & Partridge, 2014). The second issue is related to the lack of validity of measurements used to assess the surgical performance on these systems (Stefanidis & Heniford, 2009; Thijssen & Schijven, 2010; Våpenstad & Buzink, 2013; Stunt, Wulms, Kerkhoffs, Dankelman, van Dijk, & Tuijthof, 2014). This is perhaps due to the novelty of these systems and suggests that additional research is needed to validate these measurements.…”

Does virtual reality simulation have a role in training trauma and orthopaedic surgeons?

“…Advances in this field have prompted a rapid expansion in the number of commercially marketed surgical simulators, with more than 400 models currently available. 16 As surgical procedures can be deconstructed into a series of steps in which a learner can be trained and assessed, many simulators focus on the particular surgical skills involved in one of these steps, enabling deliberate practice of important and common aspects of procedures. These skills can be practiced efficiently until competency is acquired without exposing patients to undue risk.…”