An evaluation of statistical approaches to postmarketing surveillance

Abstract

Safety of medical products presents a serious concern worldwide. Surveillance systems of postmarket medical products have been established for continual monitoring of adverse events (AEs) in many countries, and the proliferation of electronic health record systems further facilitates continual monitoring for AEs. We review existing statistical methods for signal detection that are mostly in use in postmarketing safety surveillance of spontaneously reported AEs and we study their performance characteristics by simulation. We compare those with the likelihood ratio test (LRT) method (appropriately modified for use in pharmacovigilance) and use three different methods to generate data (AE based, drug based, and a modification of the method of Ahmed et al). Performance metrics include type I error, power, sensitivity, and false discovery rate, among others. The results show superior performance of the LRT method in almost all simulation experiments. An application to the FDA Adverse Event Reporting System database is illustrated using rhabdomyolysis‐related preferred terms reported to FDA during the third‐quarter of 2014 to the first‐quarter of 2017 for statin drugs. We present a critical discussion and recommendations for use of these methods.

Combining a Pharmacological Network Model with a Bayesian Signal Detection Algorithm to Improve the Detection of Adverse Drug Events

“…Hence, it is important to be able to detect potential ADEs using a post-market database. The traditional DPAs (IC, EBGM, ROR and PRR) showed insufficient sensitivities or specificities, resulting in false-negative or false-positive results, and their advantages and disadvantages were discussed in the recent scientific literature ( Ding, et al, 2020 ). An important finding of our study was that the top signals of the different signal detection algorithms had different patterns.…”

“…These findings were similar to those reported by Pham et al Recently, a label propagation frame based on four popular signal detection algorithms (PRR, ROR, EBGM, IC) has emerged, which constructs a drug similarity network using chemical structures and combines pre-clinical drug chemical structures with the post-market database FAERS ( Liu and Zhang, 2019 ). The different pharmacovigilance methods have been evaluated using a variety of performance metrics ( Ding, et al, 2020 ).…”

Systemic quinolones and risk of acute liver failure I: Analysis of data from the US FDA adverse event reporting system

“…In analyzing these reports, we used two of the most frequently used data mining techniques: the proportional reporting ratio (PRR) and multi‐item gamma Poisson shrinker (MGPS), 42 due to their higher sensitivity (PRR) and specificity (MGPS) for ADE signal detection 42–44 . Each technique involves comparing the proportion of ADE reports described in conjunction with each drug in the group of interest to that reported in the comparison/reference group 45,46 .…”

“Artificial Intelligence” for Pharmacovigilance: Ready for Prime Time?

“…Direct engagement with human PV experts to describe in detail how they do their work and the development of transparent and explainable ML algorithms that identify the key features and their interrelationships in achieving certain goals could converge on a detailed description of the PV cognitive framework that has long eluded the field. Currently, statistical disproportionality analyses [ 62 ] and case-series evaluations are largely separate activities. The development of a computable cognitive framework might identify ways in which traditional statistical methods can be integrated with NLP and ML algorithms to more rigorously identify unusual patterns [ 41 , 44 ] in case series.…”