Objective Toolkits are an important knowledge translation strategy for implementing digital health. We studied how toolkits for the implementation and evaluation of digital health were developed, tested, and reported. Materials and Methods We conducted a systematic review of toolkits that had been used, field tested or evaluated in practice, and published in the English language from 2009 to July 2019. We searched several electronic literature sources to identify both peer-reviewed and gray literature, and records were screened as per systematic review conventions. Results Thirteen toolkits were eventually identified, all of which were developed in North America, Europe, or Australia. All reported their intended purpose, as well as their development process. Eight of the 13 toolkits involved a literature review, 3 did not, and 2 were unclear. Twelve reported an underlying conceptual framework, theory, or model: 3 cited the normalization process theory and 3 others cited the World Health Organization and International Telecommunication Union eHealth Strategy. Seven toolkits were reportedly evaluated, but details were unavailable. Forty-three toolkits were excluded for lack of field-testing. Discussion Despite a plethora of published toolkits, few were tested, and even fewer were evaluated. Methodological rigor was of concern, as several did not include an underlying conceptual framework, literature review, or evaluation and refinement in real-world settings. Reporting was often inconsistent and unclear, and toolkits rarely reported being evaluated. Conclusion Greater attention needs to be paid to rigor and reporting when developing, evaluating, and reporting toolkits for implementing and evaluating digital health so that they can effectively function as a knowledge translation strategy.
Objective Electronic health records are increasingly utilized for observational and clinical research. Identification of cohorts using electronic health records is an important step in this process. Previous studies largely focused on the methods of cohort selection, but there is little evidence on the impact of underlying vocabularies and mappings between vocabularies used for cohort selection. We aim to compare the cohort selection performance using Australian Medicines Terminology to Anatomical Therapeutic Chemical (ATC) mappings from 2 different sources. These mappings were taken from the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM) and the Pharmaceutical Benefits Scheme (PBS) schedule. Materials and Methods We retrieved patients from the electronic Practice Based Research Network data repository using 3 ATC classification groups (A10, N02A, N06A). The retrieved patients were further verified manually and pooled to form a reference standard which was used to assess the accuracy of mappings using precision, recall, and F measure metrics. Results The OMOP-CDM mappings identified 2.6%, 15.2%, and 24.4% more drugs than the PBS mappings in the A10, N02A and N06A groups respectively. Despite this, the PBS mappings generally performed the same in cohort selection as OMOP-CDM mappings except for the N02A Opioids group, where a significantly greater number of patients were retrieved. Both mappings exhibited variable recall, but perfect precision, with all drugs found to be correctly identified. Conclusion We found that 1 of the 3 ATC groups had a significant difference and this affected cohort selection performance. Our findings highlighted that underlying terminology mappings can greatly impact cohort selection accuracy. Clinical researchers should carefully evaluate vocabulary mapping sources including methodologies used to develop those mappings.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.