Derivation and validation of a search algorithm to retrospectively identify mechanical ventilation initiation in the intensive care unit

Abstract

BackgroundThe development and validation of automated electronic medical record (EMR) search strategies are important for establishing the timing of mechanical ventilation initiation in the intensive care unit (ICU).Thus, we sought to develop and validate an automated EMR search algorithm (strategy) for time zero, the moment of mechanical ventilation initiation in the critically ill patient.MethodsThe EMR search algorithm was developed on the basis of several mechanical ventilation parameters, with the final parameter being positive end-expiratory pressure (PEEP), and was applied to a comprehensive institutional EMR database. The search algorithm was derived from a secondary retrospective analysis of a subset of 450 patients from a cohort of 2,684 patients admitted to a medical ICU and a surgical ICU from January 1, 2010, through December 31, 2011. It was then validated in an independent subset of 450 patients from the same period. The overall percent of agreement between our search algorithm and a comprehensive manual medical record review in the derivation and validation subsets, using peak inspiratory pressure (PIP) as the reference standard, was compared to assess timing of mechanical ventilation initiation.ResultsIn the derivation subset, the automated electronic search strategy achieved an 87% (κ = 0.87) perfect agreement, with 94% agreement to within one minute. In validating this search algorithm, perfect agreement was found in 92% (κ = 0.92) of patients, with 99% agreement occurring within one minute.ConclusionsThe use of an electronic search strategy resulted in highly accurate extraction of mechanical ventilation initiation in the ICU. The search algorithm of mechanical ventilation initiation is highly efficient and reliable and can facilitate both clinical research and patient care management in a timely manner.

Derivation and validation of a search algorithm to retrospectively identify CRRT initiation in the ECMO patients

“…It correctly identified 90% of patients with no difference and 100% of patients within 15 minutes of CRRT starting time. Our study results were in accordance with other recently published studies highlighting the importance of electronic search strategies in data extraction [6,7].…”

“…database analyst, medical informatics consultants or data administrators. With the widespread adoption of the use of EHR, electronic search strategies have increased in frequency [6]. Automated search strategies have shown high sensitivity and specificity in recognizing various medical entities [6,7].…”

“…With the widespread adoption of the use of EHR, electronic search strategies have increased in frequency [6]. Automated search strategies have shown high sensitivity and specificity in recognizing various medical entities [6,7]. These strategies would help to accurately identify any risk factors documented in the EHR, contribute to the utilization of various quality improvement initiatives, reduce the incidence of manual errors, and hence reduce the cost and enhance the effectiveness of medical care.…”

“…Therefore, electronic search strategies provide an added value to highly efficient data extraction within an electronic medical environment and allow rapid and more accurate analysis of large databases. [6,7].…”

The Reliability of Electronic Health Record Data Used for Obstetrical Research

“…Other areas of clinical research more frequently utilize and report a variety of statistical tests to assess the reliability of extracted EHR data. Studies commonly report (1) Cohen's kappa; [15][16][17][18][19] (2) measures of test performance, such as sensitivity 20 and specificity, 7,18,19,[21][22][23][24][25] positive predictive value (PPV), 7,19,20,23,25,26 and negative predictive value (NPV); 19,23 (3) the area under the curve (AUC); 7 (4) regression; 24,27 and (5) with simple agreement indices. 10,[28][29][30][31] Due to lack of a consistent methodology for assessment of reliability and quality of EHR data in the extant literature, we chose to use a combination of techniques: sensitivity, specificity, agreement, and Cohen's kappa.…”