Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

Abstract

Background: A plethora of methods and models of disproportionality analyses for safety surveillance have been developed to date without consensus nor a gold standard, leading to methodological heterogeneity and substantial variability in results. We hypothesized that this variability is inversely correlated to the robustness of a signal of disproportionate reporting (SDR) and could be used to improve signal detection performances.Methods: We used a validated reference set containing 399 true and false drug-event pairs and performed, with a frequentist and a Bayesian disproportionality method, seven types of analyses (model) for which the results were very unlikely to be related to actual differences in absolute risks of ADR. We calculated sensitivity, specificity and plotted ROC curves for each model. We then evaluated the predictive capacities of all models and assessed the impact of combining such models with the number of positive SDR for a given drug-event pair through binomial regression models.Results: We found considerable variability in disproportionality analysis results, both positive and negative SDR could be generated for 60% of all drug-event pairs depending on the model used whatever their truthfulness. Furthermore, using the number of positive SDR for a given drug-event pair largely improved the signal detection performances of all models.Conclusion: We therefore advocate for the pre-registration of protocols and the presentation of a set of secondary and sensitivity analyses instead of a unique result to avoid selective outcome reporting and because variability in the results may reflect the likelihood of a signal being a true adverse drug reaction.

Association Between Antiosteoporotic Drugs and Risk of Acute Kidney Injury: A Cross‐Sectional Study Using Disproportional Analysis and a Pharmacovigilance Database

“…13 ROR values were not calculated when the number of exposed cases was <3, following the recommendation of a previous report. 16 In the MDV analysis, the numbers of AKI and non-AKI cases were compared according to type of antiosteoporotic drug using Fisher's exact test. Odds ratios (ORs) and 95%CIs were calculated.…”

Editorial: Leveraging pharmacovigilance data mining with “the patient” in mind

“…The incongruence between frequentist and Bayesian disproportionality methodologies of signal detection was discussed by Liu et al and has been supported by other authors (Faich and Morris, 2012;Lee et al, 2020;Khouri et al , 2021). Ji et al explored an analytical modeling that integrates pharmacology information, such as known drug adverse event associations and other intrinsic drug properties with Bayesian disproportionality analytics.…”

Development of a Network-Based Signal Detection Tool: The COVID-19 Adversome in the FDA Adverse Event Reporting System

“…In particular, pharmacovigilance, through the so-called disproportionality approach, has attracted considerable interest by clinicians for its ability to timely and early assess emerging toxicities in recently approved drugs in different therapeutic domains ( Raschi et al, 2020a ). The strengths and limitations of these analyses are still being debated: e.g., the actual correlation of disproportionality findings with relative risk ( Khouri et al, 2021a ) and the considerable variability in results emerging from yet to settle key methodological issues ( Khouri et al, 2021b ). In the recent past, methodological efforts have mainly focused on the performance of disproportionality analyses: how to choose amongst the several spontaneous reporting systems ( Vogel et al, 2020 ), approaches and relevant thresholds ( Candore et al, 2015 ), database type and size ( Caster et al, 2020 ), as well as how to identify and account for reporting biases ( Arnaud et al, 2016 ).…”