Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

Davis, Sharon E.; Zabotka, Luke; Desai, Rishi; Wang, Shirley; Maro, Judith C.; Coughlin, Kevin; Hernández‐Muñoz, José J.; Stojanović, Dragan; Shah, Nigam H.; Smith, Joshua

doi:10.1007/s40264-023-01325-0

Cited by 12 publications

(3 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several methodological innovations can be embedded in our software. Some examples derive from the use of Electronic Health Records (EHR) that are more routinely used in pharmacoepidemiologic research 15,16 . These data bring novel challenges.…”

Section: Discussionmentioning

confidence: 99%

An open‐source implementation of tree‐based scan statistics

Russo,

Wang

2024

Pharmacoepidemiology and Drug

Self Cite

View full text Add to dashboard Cite

PurposeWe develop an open‐source R package to implement tree‐based scan statistics (TBSS) analyses.MethodsTBSS are data mining methods used by the United States Food and Drug Administration and the Centers for Disease Control. They simultaneously screen thousands of hierarchically aggregated outcomes to identify unsuspected adverse effects of drugs or vaccines, accounting for multiple comparisons. The general structure of TBSS is highly adaptable, with four essential components: (1) a hierarchical outcome structure, (2) a test statistic to be computed for each element of the hierarchy, (3) an algorithm to generate data replicates under a null distribution, and (4) observed outcomes at the lower level of the hierarchy. We encode the general TBSS framework in a convenient R package that offers user‐friendly functions for the most used TBSS methods. To illustrate the performance of our software, we evaluated two examples of archetypical TBSS analyses previously analyzed using proprietary, closed‐source TreeScan™ software. The first considers the risk of congenital malformations associated with first‐trimester exposure to valproate, and the second compares exposure to newly prescribed canagliflozin with a dipeptidyl peptidase 4 inhibitor in adults affected by type 2 diabetes.ResultsThe results of the original studies are replicated.ConclusionsThe diffusion of an open‐source implementation of TBSS can enhance innovation of TBSS methods and foster collaborations. We offer an intuitive R package implementing standard TBSS methods with accompanying tutorials. Our unified object‐oriented implementation allows expert users to extend the framework, introduce new features, or enhance existing ones.

show abstract

Section: Discussionmentioning

confidence: 99%

An open‐source implementation of tree‐based scan statistics

Russo,

Wang

2024

Pharmacoepidemiology and Drug

Self Cite

View full text Add to dashboard Cite

show abstract

“…Added to those cited by Lovell and Feldman, these limitations make still partial their usefulness. 3 Spontaneous reporting database, the approach used by our group to evaluate the potential adverse reactions of IL-23 inhibitors, is also debated by the commentary. The main function of this approach is the early identification of new, rare, and serious drug adverse reactions.…”

Section: Response To "Evaluating Adverse Events In Databases"mentioning

confidence: 99%

“…Added to those cited by Lovell and Feldman, these limitations make still partial their usefulness. 3 …”

mentioning

confidence: 99%

Response to “Evaluating Adverse Events in Databases”

Calapai,

Mannucci,

Cardia

et al. 2023

Pharmacology Res & Perspec

View full text Add to dashboard Cite

Enhancing Postmarketing Surveillance of Medical Products With Large Language Models

Matheny,

Yang,

Smith

et al. 2024

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceThe Sentinel System is a key component of the US Food and Drug Administration (FDA) postmarketing safety surveillance commitment and uses clinical health care data to conduct analyses to inform drug labeling and safety communications, FDA advisory committee meetings, and other regulatory decisions. However, observational data are frequently deemed insufficient for reliable evaluation of safety concerns owing to limitations in underlying data or methodology. Advances in large language models (LLMs) provide new opportunities to address some of these limitations. However, careful consideration is necessary for how and where LLMs can be effectively deployed for these purposes.ObservationsLLMs may provide new avenues to support signal-identification activities to identify novel adverse event signals from narrative text of electronic health records. These algorithms may be used to support epidemiologic investigations examining the causal relationship between exposure to a medical product and an adverse event through development of probabilistic phenotyping of health outcomes of interest and extraction of information related to important confounding factors. LLMs may perform like traditional natural language processing tools by annotating text with controlled vocabularies with additional tailored training activities. LLMs offer opportunities for enhancing information extraction from adverse event reports, medical literature, and other biomedical knowledge sources. There are several challenges that must be considered when leveraging LLMs for postmarket surveillance. Prompt engineering is needed to ensure that LLM-extracted associations are accurate and specific. LLMs require extensive infrastructure to use, which many health care systems lack, and this can impact diversity, equity, and inclusion, and result in obscuring significant adverse event patterns in some populations. LLMs are known to generate nonfactual statements, which could lead to false positive signals and downstream evaluation activities by the FDA and other entities, incurring substantial cost.Conclusions and RelevanceLLMs represent a novel paradigm that may facilitate generation of information to support medical product postmarket surveillance activities that have not been possible. However, additional work is required to ensure LLMs can be used in a fair and equitable manner, minimize false positive findings, and support the necessary rigor of signal detection needed for regulatory activities.

show abstract

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

Cited by 12 publications

References 96 publications

An open‐source implementation of tree‐based scan statistics

An open‐source implementation of tree‐based scan statistics

Response to “Evaluating Adverse Events in Databases”

Enhancing Postmarketing Surveillance of Medical Products With Large Language Models

Contact Info

Product

Resources

About