Physician agreement on the diagnosis of sepsis in the intensive care unit: estimation of concordance and analysis of underlying factors in a multicenter cohort

Abstract

Background Differentiating sepsis from the systemic inflammatory response syndrome (SIRS) in critical care patients is challenging, especially before serious organ damage is evident, and with variable clinical presentations of patients and variable training and experience of attending physicians. Our objective was to describe and quantify physician agreement in diagnosing SIRS or sepsis in critical care patients as a function of available clinical information, infection site, and hospital setting. Methods We conducted a post hoc analysis of previously collected data from a prospective, observational trial ( N = 249 subjects) in intensive care units at seven US hospitals, in which physicians at different stages of patient care were asked to make diagnostic calls of either SIRS, sepsis, or indeterminate, based on varying amounts of available clinical information ( clinicaltrials.gov identifier: NCT02127502). The overall percent agreement and the free-marginal, inter-observer agreement statistic kappa ( κ free ) were used to quantify agreement between evaluators (attending physicians, site investigators, external expert panelists). Logistic regression and machine learning techniques were used to search for significant variables that could explain heterogeneity within the indeterminate and SIRS patient subgroups. Results Free-marginal kappa decreased between the initial impression of the attending physician and (1) the initial impression of the site investigator ( κ free 0.68), (2) the consensus discharge diagnosis of the site investigators ( κ free 0.62), and (3) the consensus diagnosis of the external expert panel ( κ free 0.58). In contrast, agreement was greatest between the consensus discharge impression of site investigators and the consensus diagnosis of the external expert panel ( κ free 0.79). When stratified by infection site, κ free for agreement between initial and later diagnoses had a mean value + 0.24 (range − 0.29 to + 0.39) for respiratory infections, compared to + 0.70 (range + 0.42 to + 0.88) for abdominal + urinary + other infections. Bioinformatics analysis failed to clearly resolve the indeterminate diagnoses and also failed to explain why 60% of SIRS patients were treated with antibiotics. Conclusions Considerable uncertainty surrounds the differential clinical diagnosis of sepsis vs. SIRS, especially before organ damage has become highly evident, and for patients presenting with respiratory clinical signs. Our findings underscore the need to provide physicians with accurate, timely diagnostic information in evaluating possible sepsis. Electronic supplementary material ...

Serial measurement of pancreatic stone protein for the early detection of sepsis in intensive care unit patients: a prospective multicentric study

“…In the absence of an unambiguous definition of sepsis and highly accurate diagnostic tools the presence/absence of sepsis at any given days of an ICU stay was adjudicated by an expert committee. Discordant results using such approach have already been reported and vary considerably within sepsis diagnosis subgroups and clinical question [ 24 ], 25 ]. In our study not only the presence or absence of sepsis was adjudicated but also the day of event: this additional parameter contributed to the rate of discordance observed, reflecting the diagnostic challenges that clinicians are facing for the timely diagnosis of sepsis.…”

Prevalence and Clinical Characteristics of Patients With Sepsis Discharge Diagnosis Codes and Short Lengths of Stay in U.S. Hospitals

“…The ongoing common use of SIRS to diagnose sepsis was corroborated by our medical record reviews; in addition, we found that sepsis was commonly diagnosed based on elevated lactate levels or bacteremia without organ dysfunction. This underscores the inconsistent uptake of Sepsis-3 definitions as well as variability in how physicians apply sepsis diagnoses ( 3 – 5 ). Both of these factors could contribute to variability in comparing sepsis rates and outcomes across hospitals as well as potential bias in assessing sepsis trends over time using administrative data ( 22 – 24 ).…”

“…Sepsis hospitalizations are associated with substantial morbidity, mortality, and costs to the healthcare system ( 1 , 2 ). However, there is substantial variability in how sepsis is diagnosed and coded ( 3 – 5 ). “Sepsis” connotates severe illness and organ failure for many clinicians, yet a significant fraction of patients diagnosed with sepsis never require ICU care, are hospitalized for very short intervals, or are even discharged home directly from the emergency department ( 6 – 8 ).…”

Host Diagnostic Biomarkers of Infection in the ICU: Where Are We and Where Are We Going?

“…The inaccuracy of clinical gestalt in the diagnosis of sepsis in ICU patients has been demonstrated in multiple studies [ 9 – 11 ]. In a recent multicentre clinical cohort examined by Lopansri and colleagues [ 10 ], agreement between initial attending physician diagnosis of sepsis in ICU patients compared to retrospective diagnosis by an expert panel was weak (free-marginal kappa agreement 0.58). The tendency was to overdiagnose sepsis early, with ~30% of those initially determined to be septic being subsequently reclassified retrospectively.…”

“…Given the potential for poor outcomes with delayed antimicrobial initiation [ 1 ], clinicians may focus on the perceived benefit of antimicrobial administration even if the presentation may not be consistent with an infectious diagnosis—an example of base-rate neglect [ 12 ]. Clinicians tend to therefore over-estimate the true prevalence of infection [ 10 , 13 •, 14 ].…”