While data quality is recognized as a critical aspect in establishing and utilizing a CDRN, the findings from data quality assessments are largely unpublished. This paper presents a real-world account of studying and interpreting data quality findings in a pediatric CDRN, and the lessons learned could be used by other CDRNs.

Introduction:Data quality and fitness for analysis are crucial if outputs of analyses of electronic health record data or administrative claims data should be trusted by the public and the research community.Methods:We describe a data quality analysis tool (called Achilles Heel) developed by the Observational Health Data Sciences and Informatics Collaborative (OHDSI) and compare outputs from this tool as it was applied to 24 large healthcare datasets across seven different organizations.Results:We highlight 12 data quality rules that identified issues in at least 10 of the 24 datasets and provide a full set of 71 rules identified in at least one dataset. Achilles Heel is a freely available software that provides a useful starter set of data quality rules with the ability to add additional rules. We also present results of a structured email-based interview of all participating sites that collected qualitative comments about the value of Achilles Heel for data quality evaluation.Discussion:Our analysis represents the first comparison of outputs from a data quality tool that implements a fixed (but extensible) set of data quality rules. Thanks to a common data model, we were able to compare quickly multiple datasets originating from several countries in America, Europe and Asia.

Background: Clinical data research networks (CDRNs) aggregate electronic health record data from multiple hospitals to enable large-scale research. A critical operation toward building a CDRN is conducting continual evaluations to optimize data quality. The key challenges include determining the assessment coverage on big datasets, handling data variability over time, and facilitating communication with data teams. This study presents the evolution of a systematic workflow for data quality assessment in CDRNs. Implementation: Using a specific CDRN as use case, the workflow was iteratively developed and packaged into a toolkit. The resultant toolkit comprises 685 data quality checks to identify any data quality issues, procedures to reconciliate with a history of known issues, and a contemporary GitHub-based reporting mechanism for organized tracking. Results: During the first two years of network development, the toolkit assisted in discovering over 800 data characteristics and resolving over 1400 programming errors. Longitudinal analysis indicated that the variability in time to resolution (15day mean, 24day IQR) is due to the underlying cause of the issue, perceived importance of the domain, and the complexity of assessment. Conclusions: In the absence of a formalized data quality framework, CDRNs continue to face challenges in data management and query fulfillment. The proposed data quality toolkit was empirically validated on a particular network, and is publicly available for other networks. While the toolkit is user-friendly and effective, the usage statistics indicated that the data quality process is very time-intensive and sufficient resources should be dedicated for investigating problems and optimizing data for research.

Objective To give providers a better understanding of how to use the electronic health record (EHR), improve efficiency, and reduce burnout. Materials and Methods All ambulatory providers were offered at least 1 one-on-one session with an “optimizer” focusing on filling gaps in EHR knowledge and lack of customization. Success was measured using pre- and post-surveys that consisted of validated tools and homegrown questions. Only participants who returned both surveys were included in our calculations. Results Out of 1155 eligible providers, 1010 participated in optimization sessions. Pre-survey return rate was 90% (1034/1155) and post-survey was 54% (541/1010). 451 participants completed both surveys. After completing their optimization sessions, respondents reported a 26% improvement in mean knowledge of EHR functionality (P < .01), a 19% increase in the mean efficiency in the EHR (P < .01), and a 17% decrease in mean after-hours EHR usage (P < .01). Of the 401 providers asked to rate their burnout, 32% reported feelings of burnout in the pre-survey compared to 23% in the post-survey (P < .01). Providers were also likely to recommend colleagues participate in the program, with a Net Promoter Score of 41. Discussion It is possible to improve provider efficiency and feelings of burnout with a personalized optimization program. We ascribe these improvements to the one-on-one nature of our program which provides both training as well as addressing the feeling of isolation many providers feel after implementation. Conclusion It is possible to reduce burnout in ambulatory providers with personalized retraining designed to improve efficiency and knowledge of the EHR.