Subset Analysis for Screening Drug–Drug Interaction Signal Using Pharmacovigilance Database

Abstract

Many patients require multi-drug combinations, and adverse event profiles reflect not only the effects of individual drugs but also drug–drug interactions. Although there are several algorithms for detecting drug–drug interaction signals, a simple analysis model is required for early detection of adverse events. Recently, there have been reports of detecting signals of drug–drug interactions using subset analysis, but appropriate detection criterion may not have been used. In this study, we presented and verified an appropriate criterion. The data source used was the Japanese Adverse Drug Event Report (JADER) database; “hypothetical” true data were generated through a combination of signals detected by three detection algorithms. The accuracy of the signal detection of the analytic model under investigation was verified using indicators used in machine learning. The newly proposed subset analysis confirmed that the signal detection was improved, compared with signal detection in the previous subset analysis, on the basis of the indicators of Accuracy (0.584 to 0.809), Precision (= Positive predictive value; PPV) (0.302 to 0.596), Specificity (0.583 to 0.878), Youden’s index (0.170 to 0.465), F-measure (0.399 to 0.592), and Negative predictive value (NPV) (0.821 to 0.874). The previous subset analysis detected many false drug–drug interaction signals. Although the newly proposed subset analysis provides slightly lower detection accuracy for drug–drug interaction signals compared to signals compared to the Ω shrinkage measure model, the criteria used in the newly subset analysis significantly reduced the amount of falsely detected signals found in the previous subset analysis.

Improved Detection Criteria for Detecting Drug-Drug Interaction Signals Using the Proportional Reporting Ratio

“…Therefore, to verify the accuracy of CSS , we had to prepare “hypothetical” true data pertaining to adverse events. These “hypothetical” true data were also used to validate the subset analysis for detecting drug-drug interaction signals [ 12 ].…”

“…However, this study, like our previous study [ 12 ], has three limitations: The true data used in this study consist of a statistics-based drug D 1 – drug D 2 adverse event (SJS) combination rather than a pharmacology-based combination. Unfortunately, data on unknown adverse events do not exist; thus, it was necessary to use “hypothetical” true data instead of “real” true data for validation.…”

“…Therefore, SJS was the target adverse event in this study. Although this adverse event has been used previously in other comparative studies by our group [ 12 , 13 ] and other researchers [ 10 , 14 ], it is possible that different performance characteristics are obtained when different reference sets are used. Differences in the approach adopted by regulatory authorities may result in differences in the tendency to register adverse events in the spontaneous reporting system.…”

“…For example, JADER has long not accepted reports from patients, whereas the Food and Drug Administration Adverse Events Reporting System (FAERS) includes reports from non-medical professionals. It is unclear how the differences in registration tendencies would affect the results of this study [ 12 ]. However, as verified by Caster et al [ 15 ], the statistical impact of differences in the number of cases enrolled in the spontaneous reporting system on this study might be small.…”

“…The evaluation indicators that we have set were Accuracy (Equation 8 The commonality of the signals detected by each statistical model was evaluated using Cohen's kappa coefficient (κ), proportionate agreement for positive rating (P positive ), and proportionate agreement for negative rating (P negative ), as reported in previous studies [12][13][14]. In this study, we investigated the similarities with the Ω shrinkage measure for the previous/newly proposed analysis (CRR/CSS).…”