1058: Validity Evidence for Tracheostomy Skills Assessment Checklists

Poeppelman, Rachel Stork; Coles, Mary; Heater, Tom; Vohsing, Luke; Lutmer, Jeffrey; Maa, Tensing

doi:10.1097/01.ccm.0000643168.29850.c0

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A pediatric tracheostomy emergency readiness assessment tool will allow accurate assessment of hospital's and individual units' capability of responding to these events, both through in situ simulation and through debriefing clinical events. While separate instruments and algorithms exist to audit tracheostomy‐specific steps 68 for emergency management 40,69 and teamwork, 70 the tool developed in this study (Figure 3) measures both tracheostomy‐specific and systems (communication, equipment, etc) factors. Furthermore, the incorporation of diverse expert opinion for its development establishes both face and content validity.…”

Section: Discussionmentioning

confidence: 99%

Pediatric Tracheostomy Emergency Readiness Assessment Tool: International Consensus Recommendations

et al. 2023

View full text Add to dashboard Cite

ObjectiveTo achieve consensus on critical steps and create an assessment tool for actual and simulated pediatric tracheostomy emergencies that incorporates human and systems factors along with tracheostomy‐specific steps.MethodsA modified Delphi method was used. Using REDCap software, an instrument comprising 29 potential items was circulated to 171 tracheostomy and simulation experts. Consensus criteria were determined a priori with a goal of consolidating and ordering 15 to 25 final items. In the first round, items were rated as “keep” or “remove”. In the second and third rounds, experts were asked to rate the importance of each item on a 9‐point Likert scale. Items were refined in subsequent iterations based on analysis of results and respondents' comments.ResultsThe response rates were 125/171 (73.1%) for the first round, 111/125 (88.8%) for the second round, and 109/125 (87.2%) for the third round. 133 comments were incorporated. Consensus (>60% participants scoring ≥8, or mean score >7.5) was reached on 22 items distributed across three domains. There were 12, 4, and 6 items in the domains of tracheostomy‐specific steps, team and personnel factors, and equipment respectively.ConclusionsThe resultant assessment tool can be used to assess both tracheostomy‐specific steps as well as systems factors affecting hospital team response to simulated and clinical pediatric tracheostomy emergencies. The tool can also be used to guide debriefing discussions of both simulated and clinical emergencies, and to spur quality improvement initiatives.Level of EvidenceLevel 5 Laryngoscope, 2023

show abstract

Section: Discussionmentioning

confidence: 99%

Pediatric Tracheostomy Emergency Readiness Assessment Tool: International Consensus Recommendations

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Critical action steps to perform during the in situ simulations were selected by the study team through discussion and review of published international consensus recommendations for pediatric tracheostomy emergency readiness. 36 , 37 , 38 Action steps were selected with consideration to both their importance and feasibility of observation during simulation. Data definitions were developed to standardize assessment.…”

Section: Methodsmentioning

confidence: 99%

Validation of a Novel Mobile Application for Assessing Pediatric Tracheostomy Emergency Simulations

Tawfik,

Schiff,

Mosavian

et al. 2024

OTO Open

View full text Add to dashboard Cite

ObjectivePediatric tracheostomy is associated with high morbidity and mortality, yet clinician knowledge and quality of tracheostomy care may vary widely. In situ simulation is effective at detecting and mitigating related latent safety threats, but evaluation via retrospective video review has disadvantages (eg, delayed analysis, and potential data loss). We evaluated whether a novel mobile application is accurate and reliable for assessment of in situ tracheostomy emergency simulations.MethodsA novel mobile application was developed for assessment of tracheostomy emergency in situ simulation team performance. After 1.25 hours of training, 6 raters scored 10 tracheostomy emergency simulation videos for the occurrence and timing of 12 critical steps. To assess accuracy, rater scores were compared to a reference standard to determine agreement for occurrence or absence of critical steps and a timestamp within ±5 seconds. Interrater reliability was determined through Cohen's and Fleiss' kappa and intraclass correlation coefficient.ResultsRaters had 86.0% agreement with the reference standard when considering step occurrence and timing, and 92.8% agreement when considering only occurrence. The average timestamp difference from the reference standard was 1.3 ± 18.5 seconds. Overall interrater reliability was almost perfect for both step occurrence (Fleiss' kappa of 0.81) and timing of step (intraclass correlation coefficient of 0.99).DiscussionUsing our novel mobile application, raters with minimal training accurately and reliably assessed videos of tracheostomy emergency simulations and identified areas for future refinement.Implications for PracticeWith refinements, this innovative mobile application is an effective tool for real‐time data capture of time‐critical steps in in situ tracheostomy emergency simulations.

show abstract

1058: Validity Evidence for Tracheostomy Skills Assessment Checklists

Cited by 2 publications

References 0 publications

Pediatric Tracheostomy Emergency Readiness Assessment Tool: International Consensus Recommendations

Pediatric Tracheostomy Emergency Readiness Assessment Tool: International Consensus Recommendations

Validation of a Novel Mobile Application for Assessing Pediatric Tracheostomy Emergency Simulations

Contact Info

Product

Resources

About