DataGauge: A Practical Process for Systematically Designing and Implementing Quality Assessments of Repurposed Clinical Data

Abstract

The well-known hazards of repurposing data make Data Quality (DQ) assessment a vital step towards ensuring valid results regardless of analytical methods. However, there is no systematic process to implement DQ assessments for secondary uses of clinical data. This paper presents DataGauge, a systematic process for designing and implementing DQ assessments to evaluate repurposed data for a specific secondary use. DataGauge is composed of five steps: (1) Define information needs, (2) Develop a formal Data Needs Model (DNM), (3) Use the DNM and DQ theory to develop goal-specific DQ assessment requirements, (4) Extract DNM-specified data, and (5) Evaluate according to DQ requirements. DataGauge’s main contribution is integrating general DQ theory and DQ assessment methods into a systematic process. This process supports the integration and practical implementation of existing Electronic Health Record-specific DQ assessment guidelines. DataGauge also provides an initial theory-based guidance framework that ties the DNM to DQ testing methods for each DQ dimension to aid the design of DQ assessments. This framework can be augmented with existing DQ guidelines to enable systematic assessment. DataGauge sets the stage for future systematic DQ assessment research by defining an assessment process, capable of adapting to a broad range of clinical datasets and secondary uses. Defining DataGauge sets the stage for new research directions such as DQ theory integration, DQ requirements portability research, DQ assessment tool development and DQ assessment tool usability.

A method for interoperable knowledge-based data quality assessment

“…We applied our concepts in a real world use case with satisfactory results, using openCQA as our implementation. Important next steps would be to work on methods for learning of DQ-assessment knowledge, on integrating existing processes for MM elaboration, integrating existing frameworks proposing MMs [ 11 , 12 , 21 , 54 ] as well as to gain experience with collaborative governance of DQ-assessment knowledge.…”

“…As recommended for systematic DQ-assessment in the literature [ 21 ], DQ-assessment with openCQA started by precisely specifying the information of interest. The client part of openCQA (Fig.…”

“…when a multi-disciplinary project team collaboratively decides to make fields mandatory, considering a certain planned analysis (cf. [ 21 ]). The presented method can deal with an unlimited number of CIMs (archetypes and templates) for the same clinical concept.…”

“…[21]) or visual methods for DQassessment, which are common practice [22,[44][45][46]. A common problem mentioned in the literature is that domain experts are often left alone with this complex and resource intensive task, ending up in single-use projectspecific MMs [21,22]. Two of our concepts address this: First, automatic generation of MMs depending on the variable's reference model type and second, the knowledge-based approach.…”

“…Changes to MMs governed separately from the applying software do not require changes to the software’s source code, thus shifting the ability for MM-governance away from software developers towards domain experts. In a recent work on a systematic DQ-assessment process Diaz-Garelli et al [ 21 ] stress that adapting DQ-assessment to the task at hand is important, but defining DQ-requirements is a complex and resource-intensive task, typically requiring a multidisciplinary team. Governance of MMs in a knowledge-based approach could support this multidisciplinary collaboration.…”