Is the quality of hospital EHR data sufficient to evidence its ICHOM outcomes performance in heart failure? A pilot evaluation

Abstract

There is increasing recognition that healthcare providers need to focus attention, and be judged against, the impact they have on the health outcomes experienced by patients. The measurement of health outcomes as a routine part of clinical documentation is probably the only scalable way of collecting outcomes evidence, since secondary data collection is expensive and error prone. However, there is uncertainty about whether routinely collected clinical data within EHR systems includes the data most relevant to measuring and comparing outcomes, and if those items are collected to a good enough data quality to be relied upon for outcomes assessment, since several studies have pointed out significant issues regarding EHR data availability and quality. In this paper, we first describe a practical approach to data quality assessment of health outcomes, based on a literature review of existing frameworks for quality assessment of health data and multi-stakeholder consultation. Adopting this approach, we perform a pilot study on a subset of 21 International Consortium for Health Outcomes Measurement (ICHOM) outcomes data items from patients with congestive heart failure. To this end, all available registries compatible with the diagnosis of heart failure within the IMASIS-2 data repository connected to the Hospital del Mar network (142,345 visits of 12,503 patients) were extracted and mapped to the ICHOM format. We focus our pilot assessment on five commonly used data quality dimensions: completeness, correctness, consistency, uniqueness and temporal stability. Overall, this pilot study reveals high scores on the consistency, completeness and uniqueness dimensions. Temporal stability analyses show some changes over time in the reported use of medication to treat heart failure, as well as in the recording of past medical conditions. Finally, investigation of data correctness suggests several issues concerning the proper characterization of missing data values. Many of these issues appear to be introduced while mapping the IMASIS-2 relational database contents to the ICHOM format, as the latter requires a level of detail which is not explicitly available in the coded data of an EHR. To truly examine to what extent hospitals today are able to routinely collect the evidence of their success in achieving good health outcomes, future research would benefit from performing more extensive data quality assessments, including all data items from the ICHOM heart failure standard set, across multiple hospitals.

Frameworks, Dimensions, Definitions of Aspects, and Assessment Methods for the Appraisal of Quality of Health Data for Secondary Use: Comprehensive Overview of Reviews

“…Furthermore, this overview focused on published reviews. Important information can also be found in grey literature [ 51 , 52 ] and in studies that collect stakeholders’ opinions on the quality of health data [ 20 ]. Finally, none of the included reviews discussed patient-generated data or data generated by wearables.…”

“…To bring clarity to the diverse dimensions and definitions scattered throughout the literature, we labeled the observed definitions of dimensions from the reviews as “aspects.” We then used the framework of the i~HD. This framework underwent extensive validation through a large-scale exercise and was published [ 20 ]. It will now serve as a reference framework for mapping the diverse literature in the field.…”

“…It will now serve as a reference framework for mapping the diverse literature in the field. This overarching framework comprised 9 loosely delineated data quality dimensions ( Textbox 2 , [ 20 ]). Each observed definition of a data quality dimension was mapped onto a dimension of this reference framework.…”

“…This framework was the result of 2 stakeholder meetings in which data quality terms and definitions were grouped into an overall conceptual framework. The i~HD (European Institute for Innovation through Health Data) prioritized 9 data quality dimensions as most important to assess the quality of health data [ 20 ]. These dimensions were selected during a series of workshops with clinical care, clinical research, and ICT leads from 70 European hospitals.…”

Building a Foundation for High-Quality Health Data: A Multi-Hospital Study in Belgium (Preprint)

“…In this case study the data quality framework developed by the European Institute for Innovation through Health data (i~HD) was adopted [8,22]. This data quality framework is the prior result of analysing 22 different published frameworks and consolidating all of their defined data quality dimensions into a consolidated framework, condensed into nine distinct data quality dimensions.…”