Cleft Lip Repair Competence Can Be Evaluated with High-fidelity Simulation

Abstract

Background: Demonstrating competency before independent practice is increasingly important in surgery. This study tests the hypothesis that a high-fidelity cleft lip simulator can be used to discriminate performance between training levels, demonstrating its utility for assessing procedural competence. Methods: During this prospective cohort study, participants performed a unilateral cleft lip repair on a high-fidelity simulator. Videos were blindly rated using the Objective Structured Assessment of Technical Skills (OSATS) and the Unilateral Cleft Lip Repair Competency Assessment Tool (UCLR). Digital measurement of symmetry was estimated. Influence of training level and cumulative prior experience on each score was estimated using Pearson r. Results: Participants (n = 26) ranged from postgraduate year 3 to craniofacial fellow. Training level correlated best with UCLR (R = 0.4842, P = 0.0122*) and more weakly with OSATS (R = 0.3645, P = 0.0671), whereas cumulative prior experience only weakly correlated with UCLR (R = 0.3450, P = 0.0843) and not with OSATS (R = 0.1609, P = 0.4323). UCLR subscores indicated marking the repair had little correlation with training level (R = 0.2802, P = 0.1656), whereas performance and result did (R = 0.5152, P = 0.0071*, R = 0.4226, P = 0.0315*, respectively). Correlation between symmetry measures and training level was weak.Conclusions: High-fidelity simulation paired with an appropriate procedure-specific assessment tool has the construct validity to evaluate performance for cleft lip repair. Simply being able to mark a cleft lip repair is not an accurate independent assessment method nor is symmetry of the final result.

Teaching Unilateral Cleft lip Repair: Lessons from Simulation-Based Mining of Trainee Strengths and Performance Gaps

“…Finally, participants scored poorest on items of the "Result" subscale (1.67-2.25 mean score). Although in our previous study, the aggregate "Results" score as a whole correlated with training level (R = 0.4226, p = 0.0315), 1 in the current subanalysis of individual items, only "symmetric appearance of Cupid's bow" had an independent significant correlation with training level (R = 0.3981, p = 0.0440). However, it is worth noting that correlation for "continuity of the vermilion cutaneous junction" and "symmetry of the nasal tip" both trended toward significant correlation and the "Results" items generally correlated with RMSD as detailed in Table 1 and Figure 4.…”

“…In a pilot study, our team constructed and preliminarily validated a high-fidelity cleft lip simulator by comparing user experience and pre/post-operative form to real patients. 2 More recent studies have demonstrated the simulator's suitability for discrimination between training levels 1 and are investigating its impact on long-term learning. Conducting these studies raised new questions about what the findings could tell us about our efficacy as educators and where efforts should be focused to improve how cleft lip repair is taught.…”

“…Secondary analysis of simulation data collected for a previous study was performed providing a more granular analysis of participants’ performance during an initial uncoached cleft lip repair on a high-fidelity simulator. 1,2 This research was approved by the Institutional Review Board – (IRB-P00035608) and was conducted according to the World Medical Association Declaration of Helsinki.…”

“…Next, the learner would observe an expert perform the procedure and then, in a graduated fashion, assist with the procedure and gain experience independently performing specific aspects prior to completion of training. The results of our recently published 1 cleft lip simulation study caused us to scrutinize the effectiveness of the traditional cleft training approach revealing focused targets of educational redesign.…”

“…We recently evaluated the utility of simulation as a safe, structured and "on-demand" approach to teach and assess competency in cleft lip repair for surgical trainees. 1 During this study, investigators made unexpected observations while rating videos of residents and fellows performing cleft lip repair on the simulator. First, some parts of the cleft lip repair procedure, such as pre-operative markings, were done relatively well across all post-graduate years (PGYs).…”