The Emergency Department Trigger Tool: Validation and Testing to Optimize Yield

Abstract

Objective: Recognized as a premier approach for adverse event (AE) detection, trigger tools have been developed for multiple clinical settings outside the emergency department (ED). We recently derived and tested an ED trigger tool (EDTT) with enhanced features for high-yield detection of harm, consisting of 30 triggers associated with AEs. In this study, we validate the EDTT in an independent sample and compare record selection approaches to optimize yield for quality improvement. Methods: This is a retrospective observational study using data from 13 months of visits to an urban, academic ED by patients aged ≥ 18 years (92,859 records). We conducted standard two-tiered trigger tool reviews on an independent validation sample of 3,724 records with at least one of the 30 triggers found associated with AEs in our previous derivation sample (N = 1,786). We also tested three new candidate triggers and reviewed 72 records with no triggers for comparison purposes. We compare derivation and validation samples on: 1) triggers showing persistent associations with AEs, 2) AE yield (AEs detected/records reviewed), and 3) representativeness of AE types detected. We use bivariate associations of triggers with AEs as the basis for trigger selection. We then use multivariable modeling in the combined derivation and validation samples to determine AE risk scores using trigger weights. This allows us to predict occurrence of AEs and derive population prevalence estimates. Finally, we compare yield for detection of AEs under three record selection strategies (random selection, trigger counts, weighted trigger counts). Results: Twenty-four of the 30 triggers were confirmed to be associated with AEs on bivariate testing. Three previously marginal triggers and two of three new candidate triggers were also found to be associated with AEs. The presence of any of these 29 triggers was associated with an AE rate of 10% in our selected sample (compared to 1.1% for none, p < 0.001). The risk of an AE increased with number of triggers. Combining data from both phases, we identified 461 AEs in 429 unique visits in 5,582 records reviewed. Our multivariable model (which emphasized parsimony) retained 12 triggers with a ROC AUC of 82% in both samples. Selecting records for review based on number of triggers improves yield to 14% for 4+ triggers (top 10% of visits) and to 28% for 8+ (top 1%). A weighted trigger count has corresponding yields of 18 and 38%. The method for selecting records for review did not appear to affect event-type representativeness, with similar distributions of event types and severities detected. Conclusions: In this single-site study of the EDTT we observed high levels of validity in trigger selection, yield, and representativeness of AEs, with yields that are superior to estimates for traditional approaches to AE detection. Record selection using weighted triggers outperforms a trigger count threshold approach and far outperforms random sampling from records with at least one trigger. The EDTT is a promising efficient ...

Validation and Optimization of the Trigger Tool for the Detection of Adverse Events in General Surgery.

“…The prevalence of AE detected in our study is greater than that reported in studies on AE 17 but similar to that reported in studies where the trigger methodology was used in 7-40% of hospitalized patients 20 .…”

“…For some of them the nal model included those with a PPV greater than 5% , 19 . In others a subsequent study was performed for its validation by means of calculating false negatives in a random sample 20 . Some works report the review of trigger histories.…”

“…However, there was a special focus on training reviewers and homogenization of criteria with close tutoring by the research team. 12 ,Erlinda Daniela Padilla Zegarra 13 , Mario Franco Chacon 13 , Robert Memba Ikuga 13 , Rosa Joroba Martin 13 , Fernando Alcaide Matas 14 , paula Troncoso Pereira 14 , Víctor Soria Aledo 15 , Carmen Victoria Pérez Guarinos 15 , Sixto Genzor Rios 16 , Miguel Ángel Dobón Rascón 16 , Sandra Núñez Fernández 17 , Ernesta Valerias Domínguez 17 , Manuel García García 17 , Vanesa Zambrana Campos 18 , Pere Rebasa Cladera 19 , Mariano Artés Caselles 20 , Matías Cea Soriano 20 , Daniel Gambí Pisonero 21 , Santos Jiménez de los Galanes 22 , Maria Dolores Frutos 23…”

Predictive Power of the "Trigger Tool" for the detection of adverse events in general surgery: a multicenter observational validation study

“…Regarding the different validation methodologies of the “Trigger Tool”, it should be noted that several studies have been performed in other specialties [ 17 – 19 ]. Some works have also published results on optimization of the tool in different areas.…”

“…For some of them the final model included those with a PPV greater than 5% [ 18 , 21 ]. In others a subsequent study was performed for its validation by means of calculating false negatives in a random sample [ 19 ].…”