A Comparison Study of Algorithms to Detect Drug–Adverse Event Associations: Frequentist, Bayesian, and Machine-Learning Approaches

Pham, Minh; Cheng, Feng; Ramachandran, Kandethody M.

doi:10.1007/s40264-018-00792-0

Cited by 43 publications

(41 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although this study did not primarily aim to compare the performance of disproportionality analysis models and methods, it supplements the knowledge on this topic. Importantly, we observed very similar results between frequentist and Bayesian methods, in accordance with some previous studies (Candore et al, 2015;Pham et al, 2019). This might indicate that our results could be generalized to other methods.…”

Section: Discussionsupporting

confidence: 92%

“…sex, age, underlying conditions) (Raschi et al, 2018;Sandberg et al, 2020). Several studies have assessed and compared the performances of such methods and models, which did not reveal significant differences for signal detection (Harpaz et al, 2013;Candore et al, 2015;Pham et al, 2019). As a result, no consensus exists to date on the best analyses and no gold standard has been defined (Wisniewski et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

et al. 2021

View full text Add to dashboard Cite

Background: A plethora of methods and models of disproportionality analyses for safety surveillance have been developed to date without consensus nor a gold standard, leading to methodological heterogeneity and substantial variability in results. We hypothesized that this variability is inversely correlated to the robustness of a signal of disproportionate reporting (SDR) and could be used to improve signal detection performances.Methods: We used a validated reference set containing 399 true and false drug-event pairs and performed, with a frequentist and a Bayesian disproportionality method, seven types of analyses (model) for which the results were very unlikely to be related to actual differences in absolute risks of ADR. We calculated sensitivity, specificity and plotted ROC curves for each model. We then evaluated the predictive capacities of all models and assessed the impact of combining such models with the number of positive SDR for a given drug-event pair through binomial regression models.Results: We found considerable variability in disproportionality analysis results, both positive and negative SDR could be generated for 60% of all drug-event pairs depending on the model used whatever their truthfulness. Furthermore, using the number of positive SDR for a given drug-event pair largely improved the signal detection performances of all models.Conclusion: We therefore advocate for the pre-registration of protocols and the presentation of a set of secondary and sensitivity analyses instead of a unique result to avoid selective outcome reporting and because variability in the results may reflect the likelihood of a signal being a true adverse drug reaction.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In the bivariate disproportionality analysis, the frequentist methods generally have the following advantages and limitations compared with Bayesian methods. Several comparative studies of detection trends of these detection approaches have been reported (van, Puijenbroek et al, 2002; Kubota et al, 2004; Li et al, 2008; Bonneterre et al, 2012; Ang et al, 2016; Pham et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

Review of Statistical Methodologies for Detecting Drug–Drug Interactions Using Spontaneous Reporting Systems

2019

View full text Add to dashboard Cite

Concomitant use of multiple drugs for therapeutic purposes is known as “polypharmacy situations,” which has been recognized as an important social problem recently. In polypharmacy situations, each drug not only induces adverse events (AEs) but also increases the risk of AEs due to drug–drug interactions (DDIs). The proportion of AEs caused by DDIs is estimated to be around 30% of unexpected AEs. The randomized clinical trials in pre-marketing typically focus emphasis on the verification of single drug safety and efficacy rather than the surveys of DDI, and therefore, patients on multiple drugs are usually excluded. However, unlike pre-marketing randomized clinical trials, in clinical practice (= post marketing), many patients use multiple drugs. The spontaneous reporting system is one of the significant sources drug safety surveillance in post-marketing. Commonly, signals of potential drug-induced AEs detected from this source are validated in real-world settings. Recently, not only methodological studies on signal detection of “single” drug, but also on several methodological studies on signal detection of DDIs have been conducted. On the other hand, there are few articles that systematically summarize the statistical methodology for signal detection of DDIs. Therefore, this article reviews the studies on the latest statistical methodologies from classical methodologies for signal detection of DDIs using spontaneous reporting system. This article describes how to calculate for each detection method and the major findings from the published literatures about DDIs. Finally, this article presented several limitations related to the currently used methodologies for signal detection of DDIs and suggestions for further studies.

show abstract

“…The method assumes that each AE in each case report is caused by only one drug and then iteratively modifies the effective samples based on GPS signals (expectation step) and recalculates the GPS signals (maximization step) [ 28 ]. An independent comparison study showed that MCEM had the second highest area under the receiver operating characteristics (ROC) curve (AUC) and the highest Youden’s index [ 37 ] compared with other traditional disproportionality methods and performed very well in terms of high specificity based on its data set and evaluation strategy [ 38 ]. We used this method to generate signals from FAERS from 2012 to 2014.…”

Section: Methodsmentioning

confidence: 99%

Combining Social Media and FDA Adverse Event Reporting System to Detect Adverse Drug Reactions

2020

View full text Add to dashboard Cite

Introduction Adverse drug reactions (ADRs) are unintended reactions caused by a drug or combination of drugs taken by a patient. The current safety surveillance system relies on spontaneous reporting systems (SRSs) and more recently on observational health data; however, ADR detection may be delayed and lack geographic diversity. The broad scope of social media conversations, such as those on Twitter, can include health-related topics. Consequently, these data could be used to detect potentially novel ADRs with less latency. Although research regarding ADR detection using social media has made progress, findings are based on single information sources, and no study has yet integrated drug safety evidence from both an SRS and Twitter. Objective The aim of this study was to combine signals from an SRS and Twitter to facilitate the detection of safety signals and compare the performance of the combined system with signals generated by individual data sources. Methods We extracted potential drug–ADR posts from Twitter, used Monte Carlo expectation maximization to generate drug safety signals from both the US FDA Adverse Event Reporting System and posts from Twitter, and then integrated these signals using a Bayesian hierarchical model. The results from the integrated system and two individual sources were evaluated using a reference standard derived from drug labels. Area under the receiver operating characteristics curve (AUC) was computed to measure performance. Results We observed a significant improvement in the AUC of the combined system when comparing it with Twitter alone, and no improvement when comparing with the SRS alone. The AUCs ranged from 0.587 to 0.637 for the combined SRS and Twitter, from 0.525 to 0.534 for Twitter alone, and from 0.612 to 0.642 for the SRS alone. The results varied because different preprocessing procedures were applied to Twitter. Conclusion The accuracy of signal detection using social media can be improved by combining signals with those from SRSs. However, the combined system cannot achieve better AUC performance than data from FAERS alone, which may indicate that Twitter data are not ready to be integrated into a purely data-driven combination system.

show abstract

A Comparison Study of Algorithms to Detect Drug–Adverse Event Associations: Frequentist, Bayesian, and Machine-Learning Approaches

Cited by 43 publications

References 23 publications

Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

Review of Statistical Methodologies for Detecting Drug–Drug Interactions Using Spontaneous Reporting Systems

Combining Social Media and FDA Adverse Event Reporting System to Detect Adverse Drug Reactions

Contact Info

Product

Resources

About