Automating Quality Metrics in the Era of Electronic Medical Records: Digital Signatures for Ventilator Bundle Compliance

Abstract

Ventilator-associated events (VAEs) are associated with increased risk of poor outcomes, including death. Bundle practices including thromboembolism prophylaxis, stress ulcer prophylaxis, oral care, and daily sedation breaks and spontaneous breathing trials aim to reduce rates of VAEs and are endorsed as quality metrics in the intensive care units. We sought to create electronic search algorithms (digital signatures) to evaluate compliance with ventilator bundle components as the first step in a larger project evaluating the ventilator bundle effect on VAE. We developed digital signatures of bundle compliance using a retrospective cohort of 542 ICU patients from 2010 for derivation and validation and testing of signature accuracy from a cohort of random 100 patients from 2012. Accuracy was evaluated against manual chart review. Overall, digital signatures performed well, with median sensitivity of 100% (range, 94.4%–100%) and median specificity of 100% (range, 100%–99.8%). Automated ascertainment from electronic medical records accurately assesses ventilator bundle compliance and can be used for quality reporting and research in VAE.

“Bundle” Practices and Ventilator-Associated Events: Not Enough

“…We elected not to evaluate head-of-bed elevation because chart data were inconsistent. 14 We assessed compliance with each element in 24-hour windows. An eligible patient was considered to have been "exposed" to a failure the first time when that patient did not have a documented intervention in place.…”

“…The methods for deriving and validating these definitions are discussed at length elsewhere. 14 We conducted statistical analyses using JMP 10.0 software (SAS Institute, Cary, NC) and R 3.1.1 (Mavericks, Vienna, Austria). We conducted survival analysis using the R survival package version 2.38.…”

Automating Quality Measures for Heart Failure Using Natural Language Processing: A Descriptive Study in the Department of Veterans Affairs

“…Due to the increasing availability of EHRs and the development of NLP techniques, many systems and techniques have been, and continue to be, developed to encode narrative data for a variety of uses such as: assessing the incidence rates of adverse events, evaluating the success of preventive interventions, benchmark performance across hospitals, determining cardiovascular risk factors, providing smoking cessation, providing real-time quality metrics for colonoscopies (in terms of identification of adenomas and sessile serrated adenomas), developing retrospective clinical data for use in cardiovascular research using NLP, and identifying ventilator-associated events (VAEs) and quality reporting and research in VAEs [ 60 - 62 ].…”

“…Our methods complement other systems that identify hospitalized patients with HF in which machine learning approaches are used. Importantly, the complexity of implementation of these systems is well known and supports the assessment of barriers and facilitators for potential implementation [ 62 , 64 ]. The use of EHRs to automate publicly reported quality measures is receiving increasing attention, and is one of the promises of EHR implementation.…”

“…The use of NLP for quality measures also adds to the capture of large amounts of clinical data from EHRs. The next step is to transform health care big data into actionable knowledge for quality improvement and research that helps to improve patient care, and potentially limit health care costs, with the aim of developing infrastructure with real-time data to support decision making [ 62 - 64 , 66 , 67 ]. The products of this NLP pipeline could potentially impact a number of clinical areas, including personalized CDS (eg, the suggestion to administer ACEIs/ARBs when inappropriately not administered), and could both facilitate appropriate care by promoting CDS use and prevent provider fatigue by reducing the incidence of false-positive notifications [ 53 ].…”