Translating Comparative Effectiveness Research Evidence to Real-World Decision Making: Some Practical Considerations

Willke, Richard J.

doi:10.1007/978-981-10-3262-2_9

Cited by 1 publication

(1 citation statement)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In parallel to regulatory use of RWE, payers are increasingly demanding proof of real world effectiveness. Thus, insurance companies are increasingly demanding RWE to support reimbursement decisions 9 . Together, regulatory and reimbursement pathways are increasingly incorporating RWE and therefore the standard of care may soon be influenced by clinical assertions made using RWD.…”

Section: Background and Significancementioning

confidence: 99%

Real world evidence in cardiovascular medicine: ensuring data validity in electronic health record-based studies

Hernandez‐Boussard

Monda

Crespo

et al. 2019

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

Objective With growing availability of digital health data and technology, health-related studies are increasingly augmented or implemented using real world data (RWD). Recent federal initiatives promote the use of RWD to make clinical assertions that influence regulatory decision-making. Our objective was to determine whether traditional real world evidence (RWE) techniques in cardiovascular medicine achieve accuracy sufficient for credible clinical assertions, also known as “regulatory-grade” RWE. Design Retrospective observational study using electronic health records (EHR), 2010–2016. Methods A predefined set of clinical concepts was extracted from EHR structured (EHR-S) and unstructured (EHR-U) data using traditional query techniques and artificial intelligence (AI) technologies, respectively. Performance was evaluated against manually annotated cohorts using standard metrics. Accuracy was compared to pre-defined criteria for regulatory-grade. Differences in accuracy were compared using Chi-square test. Results The dataset included 10 840 clinical notes. Individual concept occurrence ranged from 194 for coronary artery bypass graft to 4502 for diabetes mellitus. In EHR-S, average recall and precision were 51.7% and 98.3%, respectively and 95.5% and 95.3% in EHR-U, respectively. For each clinical concept, EHR-S accuracy was below regulatory-grade, while EHR-U met or exceeded criteria, with the exception of medications. Conclusions Identifying an appropriate RWE approach is dependent on cohorts studied and accuracy required. In this study, recall varied greatly between EHR-S and EHR-U. Overall, EHR-S did not meet regulatory grade criteria, while EHR-U did. These results suggest that recall should be routinely measured in EHR-based studes intended for regulatory use. Furthermore, advanced data and technologies may be required to achieve regulatory grade results.

show abstract

Section: Background and Significancementioning

confidence: 99%