What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

Abstract

Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

A Systematic Approach to Reconciling Data Quality Failures: Investigation Using Spinal Cord Injury Data

“…25,38 Further, evaluating data provenance enables the reporting of data characteristics in EHR centric studies through presenting data completeness, data collection and handling, and the types of data. 36 The DQ failures identified in our data were of the typology well represented in the literature. [13][14][15][16][17][18][19][20] Missingness is a common challenge in the secondary use of EHR, and its causes include data being digitized after start of study, 14 and recording in free form clinical notes.…”

“…The questions were guided by the why, how, and who of data recording, to interpret the data in the context of its generation, and subsequent lifecycle. Data provenance has been highlighted as an important component of EHR secondary use, [35][36][37] and other studies have documented the consideration of information flow as necessary for robust secondary use and avoiding biases. 25,38 Further, evaluating data provenance enables the reporting of data characteristics in EHR centric studies through presenting data completeness, data collection and handling, and the types of data.…”

Transfer of Clinical Drug Data to a Research Infrastructure on OMOP – A FAIR Concept

“…To move from heterogenous and proprietary EHR data to OMOP that aligns to the FAIR principles [5], we developed a target oriented concept based on medical expertise and an EHR as well as an OMOP analysis (Figure 1). Working with EHR data in research requires a deep understanding of the original data (e.g data origin, data completeness, data correctness, data structure) and the given target environment for research [2]. Thus we built a multidisciplinary team of data and computer scientists as well as pharmacists.…”

“…The pandemic of coronavirus disease 2019 (COVID-19) has shown the need of standardized and reproducible research data, especially regarding drug administration, as observational studies are important to gain evidence, learn on real-word data and improve the COVID-19 patient treatment and their effects in the future [1]. However, those studies highly depend on the level of data quality, interoperability and reproducibility, even more if they are proceeded in a multi-centric environment [2].…”

Key Contributions in Clinical Research Informatics

“…High quality data sets are key assets for predicting prognosis and drug response from phenotypic, genotypic, and epigenetic data through innovative clinical trials and large-scale observational studies. However, in the rapidly growing field of real-world evidence generation, it is crucial to more critically evaluate EHR-driven studies the veracity of the data used to support the conclusions in order to prevent harm from misleading studies [15,16]. The accuracy of electronic health record data matters more than ever, especially due to the proliferation of clinical decision support, workflow systems and learning health systems.…”