A narrative review on the validity of electronic health record-based research in epidemiology

Abstract

Electronic health records (EHRs) are widely used in epidemiological research, but the validity of the results is dependent upon the assumptions made about the healthcare system, the patient, and the provider. In this review, we identify four overarching challenges in using EHR-based data for epidemiological analysis, with a particular emphasis on threats to validity. These challenges include representativeness of the EHR to a target population, the availability and interpretability of clinical and non-clinical data, and missing data at both the variable and observation levels. Each challenge reveals layers of assumptions that the epidemiologist is required to make, from the point of patient entry into the healthcare system, to the provider documenting the results of the clinical exam and follow-up of the patient longitudinally; all with the potential to bias the results of analysis of these data. Understanding the extent of as well as remediating potential biases requires a variety of methodological approaches, from traditional sensitivity analyses and validation studies, to newer techniques such as natural language processing. Beyond methods to address these challenges, it will remain crucial for epidemiologists to engage with clinicians and informaticians at their institutions to ensure data quality and accessibility by forming multidisciplinary teams around specific research projects.

Bias or biology? Importance of model interpretation in machine learning studies from electronic health records

“…Despite high specificity for accurate diagnosis of a disease, ICD codes are known to have low sensitivity; in other words the presence of a code is a likely indicator of a disease, however, the absence of a code does not reliably indicate absence of that disease. 43 The ICD-9 codes for diabetes with complications (250.1–250.9), for example, have a sensitivity of 63.6% and specificity of 99%. 44 Therefore, since a diagnosis is not captured with high probability, those with more medical encounters are more likely to have the presence of diabetes with complication detected.…”

“…Controlling for the number of inpatient encounters may have been a potential solution to remove the informed decision bias (ie, ≥ n visits to be eligible into the cohort), however, this may incur a selection bias as individuals with fewer visits would be excluded. 43 , 45 Diagnosis data have acceptable quality due to mandates requiring accurate collection of this data, and certain demographic data (ie, age, gender, and ethnicity/race) are mandated by the Meaningful Use objectives. 64 However, there is no mandated coding system for the remaining nonessential demographic (eg, income, marital status, education), laboratory, vital sign, or social data for EHRs.…”

Existing Data Sources for Clinical Epidemiology: Database of the National Hospital Organization in Japan

“…Epidemiological studies using electronic health records (EHRs) data still generally have several weaknesses and overarching challenges, not only in the NHO database: validity of data, representativeness, data availability and interpretation, and missingness. 31–33 For example, as noted in the previous section, hospital EHRs do not contain information on medical care provided outside the hospital. As a solution to this problem, based on the “Next Generation Medical Infrastructure Law” (official name “Act on Anonymized Medical Data That Are Meant to Contribute to Research and Development in the Medical Field”) which was legislated in 2018, the project has been initiated to expand the capability of the data resources by the linkage of the EHR data from the NHO database and the clinics across the country held by the Japan Medical Association.…”

Challenges and recommendations for high quality research using electronic health records

“…Many research protocols for access and ethics committees are not yet specifically addressing this area, despite significant attention in terms of implementation (32-34). Importantly, EHR data itself can be inherently biased, for example from the data collection process mandated by the software or if the primary use of the data is for administrative or billing purposes (6).…”

“…Researchers have extensive experience of producing high quality research from patient data, and we have worked with approval bodies which have adapted protocol guidelines to support this work. However, EHRs are different to many other sources of patient data; they are neither an opportunistic collection of existing administrative data sources nor a purposefully designed comprehensive single database (registry) (6). Rather, they collate information on the patient's clinical condition, laboratory results, diagnoses and treatments as they are experiencing health care.…”