A Comparison of Data Quality Assessment Checks in Six Data Sharing Networks

Abstract

Objective: To compare rule-based data quality (DQ) assessment approaches across multiple national clinical data sharing organizations. Methods:Six organizations with established data quality assessment (DQA) programs provided documentation or source code describing current DQ checks. DQ checks were mapped to the categories within the data verification context of the harmonized DQA terminology. To ensure all DQ checks were consistently mapped, conventions were developed and four iterations of mapping performed. Difficultto-map DQ checks were discussed with research team members until consensus was achieved.Results: Participating organizations provided 11,026 DQ checks, of which 99.97 percent were successfully mapped to a DQA category. Of the mapped DQ checks (N=11,023), 214 (1.94 percent) mapped to multiple DQA categories. The majority of DQ checks mapped to Atemporal Plausibility (49.60 percent), Value Conformance (17.84 percent), and Atemporal Completeness (12.98 percent) categories. Discussion: Using the common DQA terminology, near-complete (99.97 percent) coverage across a wide range of DQA programs and specifications was reached. Comparing the distributions of mapped DQ checks revealed important differences between participating organizations. This variation may be related to the organization's stakeholder requirements, primary analytical focus, or maturity of their DQA program. Not within scope, mapping checks within the data validation context of the terminology may provide additional insights into DQA practice differences. Conclusion:A common DQA terminology provides a means to help organizations and researchers understand the coverage of their current DQA efforts as well as highlight potential areas for additional DQA development. Sharing DQ checks between organizations could help expand the scope of DQA across clinical data networks.

Development and evaluation of a computable phenotype to identify pediatric patients with leukemia and lymphoma treated with chemotherapy using electronic health record data

“…In PEDSnet, this corresponded to specialty codes representing one of the following: “pediatric hematology‐oncology,” “medical oncology,” and “hematology.” A patient was also considered to have seen a hematologist‐oncologist if a clinic encounter occurred in a clinic identified by the hospital as providing hematology‐oncology care. PEDSnet performs data quality analysis quarterly with approximately 850 data quality checks that monitor for fidelity, consistency, accuracy, and completeness. PEDSnet publishes the types of errors detected online.…”

Quality Informatics: The Convergence of Healthcare Data, Analytics, and Clinical Excellence

“…Data-quality assessment frameworks are being formalized for characterizing healthcare data quality for secondary uses. [24][25][26][27][28][29][30] Notable challenges include development of metrics to assess data quality, 24 definition of completeness users choose to use, 25 and potential biases in datasets. 26 Challenges facing systems and their attempts to use data for quality improvement include usability, interoperability, full integration, and data mining.…”

Evaluating Foundational Data Quality in the National Patient-Centered Clinical Research Network (PCORnet®)

“…DRNs have developed a variety of processes to evaluate the quality of the data in their networks [14161718]. In PCORnet, data quality assessments encompass a broad range of activities including those conducted by individual institutions, the CDRNs [19], and the Coordinating Center.…”