Rare Diseases in Hospital Information Systems—An Interoperable Methodology for Distributed Data Quality Assessments

Abstract

Background: Multisite research networks such as the project “Collaboration on Rare Diseases” connect various hospitals to obtain sufficient data for clinical research. However, data quality (DQ) remains a challenge for the secondary use of data recorded in different health information systems. High levels of DQ as well as appropriate quality assessment methods are needed to support the reuse of such distributed data. Objectives: The aim of this work is the development of an interoperable methodology for assessing the quality of data recorded in heterogeneous sources to improve the quality of rare disease (RD) documentation and support clinical research. Methods: We first developed a conceptual framework for DQ assessment. Using this theoretical guidance, we implemented a software framework that provides appropriate tools for calculating DQ metrics and for generating local as well as cross-institutional reports. We further applied our methodology on synthetic data distributed across multiple hospitals using Personal Health Train. Finally, we used precision and recall as metrics to validate our implementation. Results: Four DQ dimensions were defined and represented as disjunct ontological categories. Based on these top dimensions, 9 DQ concepts, 10 DQ indicators, and 25 DQ parameters were developed and applied to different data sets. Randomly introduced DQ issues were all identified and reported automatically. The generated reports show the resulting DQ indicators and detected DQ issues. Conclusion: We have shown that our approach yields promising results, which can be used for local and cross-institutional DQ assessments. The developed frameworks provide useful methods for interoperable and privacy-preserving assessments of DQ that meet the specified requirements. This study has demonstrated that our methodology is capable of detecting DQ issues such as ambiguity or implausibility of coded diagnoses. It can be used for DQ benchmarking to improve the quality of RD documentation and to support clinical research on distributed data.

High-Quality Data for Health Care and Health Research

“…9 With “consistency comprises indicators that use Boolean type checks to identify inadmissible, impossible, or uncertain data values or combinations of data values,” Yusuf et al cite an orthogonal perspective on this dimension. 11 Tahar et al decline the use of the term “consistency” for their study and refer to plausibility instead, broadly defined as “deviations from expected values.” 14 Smith et al talk about “consistency and similarity of synthetic samples” without precisely mentioning their interpretation of the dimension. 16 Beyond the different interpretations of terms that are directly related to the topic data quality, one might imagine similar deviations regarding basic concepts such as “data set” or “metadata.” Tahar et al looked at those terms and propose respective definitions.…”

“…using synthetic data. 14 As a unique contribution to this focus theme, the authors provide definitions of basic concepts as "data item" carefully distinguishing between the levels of data and metadata. The authors demonstrated the applicability of their approach interconnecting three university hospitals with a Personal Health Train.…”

“…11 Tahar et al decline the use of the term "consistency" for their study and refer to plausibility instead, broadly defined as "deviations from expected values". 14 Smith et al talk about "consistency and similarity of synthetic samples" without precisely mentioning their interpretation of the dimension. 16 Beyond the different interpretation of terms that are directly related to the topic data quality, one might imagine similar deviations regarding basic concepts such as "data set" or "metadata".…”

“…13 The sixth paper transferred the methodology of data quality assessments to distributed sources. 14 The two remaining papers picked up the approach of synthetic data collections. 15,16 Concepts Related to Data Quality Mashoufi et al present a scoping review about the main concepts related to data quality and data quality assessment methodologies.…”

“…Beyond the different interpretations of terms that are directly related to the topic data quality, one might imagine similar deviations regarding basic concepts such as "data set" or "metadata." Tahar et al looked at those terms and propose respective definitions 14. Nevertheless, the manifold perspectives and interpretations of domainspecific and basic terms will impede the distribution of the concepts, frameworks, packages, and tools presented by the papers to a wider application.…”