Verification of the “Upward Variation in the Reporting Odds Ratio Scores” to Detect the Signals of Drug–Drug Interactions

Abstract

The reporting odds ratio (ROR) is easy to calculate, and there have been several examples of its use because of its potential to speed up the detection of drug–drug interaction signals by using the “upward variation of ROR score.” However, since the validity of the detection method is unknown, this study followed previous studies to investigate the detection trend. The statistics models (the Ω shrinkage measure and the “upward variation of ROR score”) were compared using the verification dataset created from the Japanese Adverse Drug Event Report database (JADER). The drugs registered as “suspect drugs” in the verification dataset were considered as the drugs to be investigated, and the target adverse event in this study was Stevens–Johnson syndrome (SJS), as in previous studies. Of 3924 pairs that reported SJS, the number of positive signals detected by the Ω shrinkage measure and the “upward variation of ROR score” (Model 1, the Susuta Model, and Model 2) was 712, 2,112, 1758, and 637, respectively. Furthermore, 1239 positive signals were detected when the Haldane–Anscombe 1/2 correction was applied to Model 2, the statistical model that showed the most conservative detection trend. This result indicated the instability of the positive signal detected in Model 2. The ROR scores based on the frequency-based statistics are easily inflated; thus, the use of the “upward variation of ROR scores” to search for drug–drug interaction signals increases the likelihood of false-positive signal detection. Consequently, the active use of the “upward variation of ROR scores” is not recommended, despite the existence of the Ω shrinkage measure, which shows a conservative detection trend.

Association between road traffic accidents and drugs belonging to the antiseizure medications class: A pharmacovigilance analysis in VigiBase

“…Since as many models as the number of pairs of ASM were used, a Bonferroni correction was used to prevent from α risk inflation in DDI analyses, and results were presented with corrected 95% CI. As ROR scores can be inflated for combinations with fewer reports of interaction signals, 20 we decided to perform a sensitivity interaction analysis using the information component, with a similar Bonferroni correction. In disproportionality analysis, positive controls are drugs established to provoke the AE of interest.…”

Comment on: ‘Detecting drug‐drug interactions that increase the incidence of long QT syndrome using a spontaneous reporting system’ by Matsuo et al

“…However, we recently reported that the ROR scores based on the frequency‐based statistics (definition by Susuta et al 4 ) are easily inflated; thus, the use of the “upward variation of ROR scores” to search for DDI signals increases the likelihood of false‐positive signal detection 5 . This means that the validity of the DDI signal reported by Matsuo et al is unclear.…”

Comment on: “Adverse reactions associated with immune checkpoint inhibitors and bevacizumab: A pharmacovigilance analysis”

“…Although many algorithms for exploring safety signals have been reported, 3,4 the ROR is used to calculate safety signal scores for single agents, not recommended for combination therapies. 5 The Ω shrinkage measure proposed by Norén et al 6 is used by the World Health Organization Uppsala Monitoring Center (WHO-UMC), 7 and the previous study has shown that it has the most conservative signal detection trend among signal detection methods based on frequentist statistics. 8 Since the original article 1 does not provide detailed data on the number of reports, I could not calculate Ω shrinkage measure scores and validate the ROR score for the DDI signals between ICIs and bevacizumab.…”

“…I reported that the ROR scores based on the frequencybased statistics were easily inflated; thus, using the "upward variation of ROR scores" to search for DDI signals increases the likelihood of false-positive signal detection. 5 Since the "odds ratio of combination therapy to monotherapy" in the original article approximates the "upward variation score of the ROR score," it is difficult to exclude the possibility that false-positive signals are included. They perform multivariate logistic regression (MLR) analysis to ascertain whether combination therapy is an independent risk factor for ADRs.…”