Determining the Electronic Signature of Infection in Electronic Health Record Data

Abstract

OBJECTIVES: Recent sepsis studies have defined patients as “infected” using a combination of culture and antibiotic orders rather than billing data. However, the accuracy of these definitions is unclear. We aimed to compare the accuracy of different established criteria for identifying infected patients using detailed chart review. DESIGN: Retrospective observational study. SETTING: Six hospitals from three health systems in Illinois. PATIENTS: Adult admissions with blood culture or antibiotic orders, or Angus International Classification of Diseases infection codes and death were eligible for study inclusion as potentially infected patients. Nine-hundred to 1,000 of these admissions were randomly selected from each health system for chart review, and a proportional number of patients who did not meet chart review eligibility criteria were also included and deemed not infected. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The accuracy of published billing code criteria by Angus et al and electronic health record criteria by Rhee et al and Seymour et al (Sepsis-3) was determined using the manual chart review results as the gold standard. A total of 5,215 patients were included, with 2,874 encounters analyzed via chart review and a proportional 2,341 added who did not meet chart review eligibility criteria. In the study cohort, 27.5% of admissions had at least one infection. This was most similar to the percentage of admissions with blood culture orders (26.8%), Angus infection criteria (28.7%), and the Sepsis-3 criteria (30.4%). Sepsis-3 criteria was the most sensitive (81%), followed by Angus (77%) and Rhee (52%), while Rhee (97%) and Angus (90%) were more specific than the Sepsis-3 criteria (89%). Results were similar for patients with organ dysfunction during their admission. CONCLUSIONS: Published criteria have a wide range of accuracy for identifying infected patients, with the Sepsis-3 criteria being the most sensitive and Rhee criteria being the most specific. These findings have important implications for studies investigating the burden of sepsis on a local and national level.

Trends in the incidence and outcome of sepsis using data from a Japanese nationwide medical claims database-the Japan Sepsis Alliance (JaSA) study group-

“…Since the definition of sepsis requires laboratory data, including blood levels of creatinine and bilirubin and platelet counts, our extraction approach using diagnosis coding and treatment data on organ supports may underestimate the incidence compared to a more stringent extraction approach using laboratory data according to Sepsis-3. On the other hand, a recent report [ 37 ] found that the Rhee criteria [ 3 ] using a combination of culture and antibiotic orders approach is significantly less sensitive but more specific compared approaches to defining suspected infection via EHR data. We would expect the current study using suspected infection definition to be relatively specific and identify a more severely ill population.…”

Machine learning to predict vasopressin responsiveness in patients with septic shock

“…Sepsis was identified by ICD-9/ICD-10 codes, which are known to be inaccurate. 37,38 However, the familiarity and generalizability of these codes offer an advantage over automated evaluation of clinical criteria, which has also been found to be inconsistent. 39 Additionally, in identifying the study's population, inclu-…”

Identification of Clinical Phenotypes in Septic Patients Presenting With Hypotension or Elevated Lactate

“…There are advantages and disadvantages of both means of deriving study populations. EMR-based cohorts typically offer larger study populations but may have less granular data as well as decreased certainty in the diagnosis, whereas trial populations commonly have fewer subjects but certainty of sepsis or septic shock diagnosis and often highly regimented data ( 21 ).…”