Performance Characteristics of a Methodology to Quantify Adverse Events over Time in Hospitalized Patients

Abstract

Objective. To assess the performance characteristics of the Institute for Healthcare Improvement Global Trigger Tool (GTT) to determine its reliability for tracking local and national adverse event rates. Data Sources. Primary data from 2008 chart reviews. Study Design. A retrospective study in a stratified random sample of 10 North Carolina hospitals. Hospital-based (internal) and contract research organization-hired (external) reviewers used the GTT to identify adverse events in the same 10 randomly selected medical records per hospital in each quarter from January 2002 through December 2007. Data Collection/Extraction. Interrater and intrarater reliability was assessed using k statistics on 10 percent and 5 percent, respectively, of selected medical records. Additionally, experienced GTT users reviewed 10 percent of records to calculate internal and external teams' sensitivity and specificity. Principal Findings. Eighty-eight to 98 percent of the targeted 2,400 medical records were reviewed. The reliability of the GTT to detect the presence, number, and severity of adverse events varied from k 5 0.40 to 0.60. When compared with a team of experienced reviewers, the internal teams' sensitivity (49 percent) and specificity (94 percent) exceeded the external teams' (34 and 93 percent), as did their performance on all other metrics. Conclusions. The high specificity, moderate sensitivity, and favorable interrater and intrarater reliability of the GTT make it appropriate for tracking local and national adverse event rates. The strong performance of hospital-based reviewers supports their use in future studies.

Performance of the Global Assessment of Pediatric Patient Safety (GAPPS) Tool

“…We found that GAPPS had similar reliability in pediatric populations as the IHI GTT has in adult populations. 6,13 Although far more sensitive and reliable than other methods, manual review of records is time and…”

“…[8][9][10][11][12][13][14] In addition, we spoke with trigger developers, including those who developed PACHMT and the IHI GTT, to identify any additional triggers and inform GAPPS development. 7 From our review, we compiled a list of 78 candidate triggers for possible inclusion in GAPPS; this list of candidate triggers was compiled by the GAPPS team with input from the developers of PACHMT.…”

“…Patients >18 years of age and those admitted primarily for psychiatric (without a concurrent acute medical issue) or rehabilitation care were excluded, consistent with exclusions in adult trigger tool studies. 11,13 …”

Parent-Reported Errors and Adverse Events in Hospitalized Children

“…6 Error and AE detection methods have greatly advanced over the past few decades. [7][8][9][10] Medical record review-based surveillance methods using trigger tools detect errors and AEs at more than 10 times the rates of traditional incident-reporting systems or systems based on reviews of hospital administrative records. [9][10][11][12][13][14][15] While error and AE detection methods have progressed, patient and family reports remain notably absent from most surveillance systems.…”

Introductory Chapter: The Decades Long Quest Continues Toward Better, Safer Healthcare Systems

“…While no reporting system is perfect and often physicians might criticize that they are being inherently personally linked to adverse events, in a culture of shared accountability, any limitations in the data collection and the reporting process will clearly be acknowledged and used to improve our understanding of overall trends. High performing institutions might look at relative event occurrences, changes in events over time, or compare own outcomes to similar "like" institutions [32,33]. Individuals clearly should not be identiied outside of established peer-review quality initiatives.…”