Ensuring basic competency in chest tube insertion using a simulated scenario: an international validation study

Abstract

IntroductionChest tube insertion can be associated with serious complications. A structured training programme is needed to minimise complications and enhance patient safety. Novices should pass a reliable test with solid evidence of validity before performing the procedure supervised on patients. The aim of this study was to establish a credible pass/fail standard.MethodsWe used an established assessment tool the Chest Tube Insertion Competency Test (TUBE-iCOMPT). Validity evidence was explored according to Messick’s five sources of validity. Two methods were used to establish a credible pass/fail standard. Contrasting groups’ method: 34 doctors (23 novices and 11 experienced surgeons) performed the procedure twice and all procedures were video recorded, edited, blinded and rated by two independent, international raters. Modified Angoff method: seven thoracic surgeons individually determined the scores that defined the pass/fail criteria. The data was gathered in Copenhagen, Denmark and Riyadh, Saudi Arabia.ResultsInternal consistency reliability was calculated as Cronbach’s alpha to 0.94. The generalisability coefficient with two raters and two procedures was 0.91. Mean scores were 50.7 (SD±13.2) and 74.7 (SD±4.8) for novices and experienced surgeons, respectively (p<0.001). The pass/fail score of 62 points resulted in zero false negatives and only three false positives.DiscussionWe have gathered valuable additional validity evidence for the assessment tool TUBE-iCOMPT including establishment of a credible pass/fail score. The TUBE-iCOMPT can now be integrated in mastery learning programmes to ensure competency before independent practice.

Standard Setting in Simulation-based Training of Surgical Procedures

“…Seven studies used more than 1 standard setting methodology with 4 studies comparing the results of the different methodologies 29,35,57,85,86 . Concordance was found for the contrasting groups’ and Angoff methods in 3 studies 29,35,57 with the contrasting groups’ method establishing standards at a higher level than the Angoff method. Goldenberg et al 85 established a pass-fail setting for flexible uretero-renoscopy using the borderline groups’ method (examinee-based method) and compared this to a crowd-sourced method where the assessors rated a pass or fail, as well as a global rating.…”

Simulation-Based Mastery Learning Course for Tube Thoracostomy

“…The Modified TUBE-iCOMPT checklist used in this course ( Appendix A ) was adapted by the authors based on Salamonsen and colleagues’ TUBE-iCOMPT assessment ( Appendix B ), an instrument with validity evidence to assess physician skills at chest tube insertion. 11 , 12 We adapted this assessment to meet the needs of our residents in coordination with our institution's EM and trauma surgery faculty. One trauma surgery faculty member and two EM faculty members reviewed the original assessment tool and agreed on the following changes based on how the procedure was taught at our institution: The scoring sections for removing trocar (4 points) and inserting wound closure suture mid-wound (1 point) were removed, as those procedural techniques were not commonly used at our institution.…”

“…In the original validity evidence study on the TUBE-iCOMPT tool, a minimum passing score of 62 out of 84 points was established using two separate methods: contrasting groups’ standard setting and the modified Angoff method using experts from five different institutions. 12 Given that 5 of the 84 total points in the assessment were removed, the minimum passing score was lowered by 5 points to 57 on the modified rubric. In adapting this course to other institutions, we recommend reviewing the checklist to ensure consistency with an institution's standard practice.…”

Interdisciplinary teamwork for chest tube insertion and management: an integrative review

“…It is essential to promote interprofessional healthcare education and training [43]. Among the 19 studies in the systematic review, 14 only approached chest drain insertion techniques for physicians or medical students [4,[21][22][23][24][25][26][27][28][29][30][31][32][33]. None studied interdisciplinary work in drain insertion or chest drain care, and only 1 presented any protocol for teaching and training students in drainage [28].…”