A Competency Assessment Tool for Unilateral Cleft Lip Repair

Abstract

Background: Objective evaluation of operative performance is increasingly important in surgical training. Evaluation tools include global rating scales of performance and procedure-specific skills checklists. For unilateral cleft lip repair, the numerous techniques make universal evaluation challenging. Thus, we sought to create a unilateral cleft lip evaluation tool agnostic to specific repair technique. Methods: Four surgeons with expertise in 3 common cleft lip repair techniques participated in a 3-round Delphi process to generate consensus evaluation points spanning all techniques. Items were categorized as marking the repair, performing the repair, and final result. Two blinded raters then scored videos of simulated cleft lip repairs using both the 21-item novel checklist and the modified Objective Structured Assessment of Technical Skills. Kappa and T values were calculated for both scales to determine level of agreement. Results: Ten videos of repairs performed by novice residents through experienced craniofacial fellows were scored. Moderate (κ = 0.41–0.60) to substantial (κ = 0.61–0.80) interrater reliability was seen for the majority of questions in both the novel tool and the Objective Structured Assessment of Technical Skills. A single question in the novel tool had almost perfect agreement (κ = 0.81–1.00), 8 had moderate agreement, and 6 had substantial agreement. Poorly scoring questions were discarded from the final 18-item tool. Conclusions: Despite variations in unilateral cleft lip repair technique, common themes exist that can be used to assess performance and outcome. A universal evaluation tool has potential implications for trainee assessment, surgeon credentialing, and screening for surgical missions.

Cleft Lip Repair Competence Can Be Evaluated with High-fidelity Simulation

“…The UCLR is an 18-item cleft lip repair skills checklist that is agnostic to eponymous repair technique with subscores for marking, performance, and result. 7 ( See appendix, Supplemental Digital Content 2, which displays the UCLR, http://links.lww.com/PRSGO/C102 .)…”

“…for marking, performance, and result. 7 (See appendix, Supplemental Digital Content 2, which displays the UCLR, http://links.lww.com/PRSGO/C102. )…”

Gender Differences in Plastic Surgery Trainee Confidence: A Pilot Analysis During Cleft Lip Simulation

“…Participants’ videos were blindly rated posthoc by a single cleft surgeon using the modified objective structured assessment of technical skills (OSATS, score range 4–20; See appendix, Supplemental Digital Content 3 , which displays the modified OSATS scale, http://links.lww.com/PRSGO/C944), a global surgical skill score, and a procedure-specific 18-item UCLR competency assessment tool (score range 18–54; See appendix, Supplemental Digital Content 4 , which displays the unilateral cleft lip repair competency assessment tool, http://links.lww.com/PRSGO/C945) 13 that is both totaled and also subdivided into “marking,” “performing,” and “results” scores.…”

Teaching Unilateral Cleft lip Repair: Lessons from Simulation-Based Mining of Trainee Strengths and Performance Gaps

“…The anonymized videos were then rated by two attending cleft surgeons using a technique-agnostic eighteen item Unilateral Cleft Lip Repair competency assessment tool (UCLR), subdivided into “Marking”, “Performing”, “Result” sections. 3 Performance on individual items of cleft lip repair was scored on a 1–3 scale (1 for “performed incorrectly or not at all,” 2 for “performed somewhat correctly,” and 3 for “performed correctly, acceptable for an attending surgeon”). This scale was designed through a Delphi-process by a group of cleft surgeons with expertise in the Millard rotation-advancement, Mohler, and Fisher unilateral cleft lip repairs to focus on common themes across all variations of cleft lip repair, rather than nuances of an individual eponymous technique.…”

“…This scale was designed through a Delphi-process by a group of cleft surgeons with expertise in the Millard rotation-advancement, Mohler, and Fisher unilateral cleft lip repairs to focus on common themes across all variations of cleft lip repair, rather than nuances of an individual eponymous technique. 3 A sensitivity analysis was performed to reveal potential scoring bias stemming from technique choice by re-analyzing the results for the most common type of repair (Mohler) by rank-ordering individual item scores and evaluating correlation with each of the variables using Pearson R. Then, we compared the results for repairs using the Mohler technique to the whole study sample. Subjective aesthetic outcome was defined by the “Results” subscore of the UCLR scale.…”