Introduction and ObjectiveSocial media has been proposed as a possibly useful data source for pharmacovigilance signal detection. This study primarily aimed to evaluate the performance of established statistical signal detection algorithms in Twitter/Facebook for a broad range of drugs and adverse events.MethodsPerformance was assessed using a reference set by Harpaz et al., consisting of 62 US Food and Drug Administration labelling changes, and an internal WEB-RADR reference set consisting of 200 validated safety signals. In total, 75 drugs were studied. Twitter/Facebook posts were retrieved for the period March 2012 to March 2015, and drugs/events were extracted from the posts. We retrieved 4.3 million and 2.0 million posts for the WEB-RADR and Harpaz drugs, respectively. Individual case reports were extracted from VigiBase for the same period. Disproportionality algorithms based on the Information Component or the Proportional Reporting Ratio and crude post/report counting were applied in Twitter/Facebook and VigiBase. Receiver operating characteristic curves were generated, and the relative timing of alerting was analysed.ResultsAcross all algorithms, the area under the receiver operating characteristic curve for Twitter/Facebook varied between 0.47 and 0.53 for the WEB-RADR reference set and between 0.48 and 0.53 for the Harpaz reference set. For VigiBase, the ranges were 0.64–0.69 and 0.55–0.67, respectively. In Twitter/Facebook, at best, 31 (16%) and four (6%) positive controls were detected prior to their index dates in the WEB-RADR and Harpaz references, respectively. In VigiBase, the corresponding numbers were 66 (33%) and 17 (27%).ConclusionsOur results clearly suggest that broad-ranging statistical signal detection in Twitter and Facebook, using currently available methods for adverse event recognition, performs poorly and cannot be recommended at the expense of other pharmacovigilance activities.Electronic supplementary materialThe online version of this article (10.1007/s40264-018-0699-2) contains supplementary material, which is available to authorized users.

Over a period of 3 years, the European Union’s Innovative Medicines Initiative WEB-RADR project has explored the value of social media (i.e., information exchanged through the internet, typically via online social networks) for identifying adverse events as well as for safety signal detection. Many patients and clinicians have taken to social media to discuss their positive and negative experiences of medications, creating a source of publicly available information that has the potential to provide insights into medicinal product safety concerns. The WEB-RADR project has developed a collaborative English language workspace for visualising and analysing social media data for a number of medicinal products. Further, novel text and data mining methods for social media analysis have been developed and evaluated. From this original research, several recommendations are presented with supporting rationale and consideration of the limitations. Recommendations for further research that extend beyond the scope of the current project are also presented.

Introduction The Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) and VigiBase ® are two established databases for safety monitoring of medicinal products, recently complemented with the EudraVigilance Data Analysis System (EVDAS). Objective Signals of disproportionate reporting (SDRs) can characterize the reporting profile of a drug, accounting for the distribution of all drugs and all events in the database. This study aims to quantify the redundancy among the three databases when characterized by two disproportionality-based analyses (DPA). Methods SDRs for 100 selected products were identified with two sets of thresholds (standard EudraVigilance SDR criteria for all vs Bayesian approach for FAERS and VigiBase ®). Per product and database, the presence or absence of SDRs was determined and compared. Adverse events were considered at three levels: MedDRA ® Preferred Term (PT), High Level Term (HLT), and HLT combined with Standardized MedDRA ® Query (SMQ). Redundancy was measured in terms of recall (SDRs in EVDAS divided by SDRs from any database) and overlap (SDRs in EVDAS and at least one other database, divided by SDRs in EVDAS). Covariates with potential impact on results were explored with linear regression models. Results The median overlap between EVDAS and FAERS or VigiBase ® was 85.0% at the PT level, 94.5% at the HLT level, and 97.7% at the HLT or SMQ level. The corresponding median recall of signals in EVDAS as a percentage of all signals generated in all three databases was 59.4%, 74.1%, and 87.9% at the PT, HLT, and HLT or SMQ levels, respectively. The overlap difference is partially explained by the relative number of EU cases in EudraVigilance and the ratio of EVDAS cases and FAERS cases, presumably due to differences in marketing authorizations, or market penetration in different regions. Products with few cases in EVDAS (< 1500) also display limited recall of signals relative to FAERs/VigiBase ®. Time-on-market does not predict signal redundancy between the three databases. The choice of the DPA has an expected but somewhat small effect on redundancy. Conclusions Organizations typically consider regulatory expectations, operating performance (e.g., positive predictive value), and procedural complexity when selecting databases for signal management. As SDRs can be seen as a proxy of general reporting characteristics identifiable in a systematic screening process, our results indicate that, for most products, these characteristics are largely similar in each of the databases.

Introduction The basis of pharmacovigilance is provided by the exchange of Individual Case Safety Reports (ICSRs) between the recipient of the original report and other interested parties, which include Marketing Authorization Holders (MAHs) and Health Authorities (HAs). Different regulators have different reporting requirements for report transmission. This results in replication of each ICSR that will exist in multiple locations. Adding in the fact that each case will go through multiple versions, different recipients may receive different case versions at different times, potentially influencing patient safety decisions and potentially amplifying or obscuring safety signals inappropriately. Objective The present study aimed to investigate the magnitude of replication, the variability among recipients, and the subsequent divergence across recipients of ICSRs. Methods Seven participating TransCelerate Member Companies (MCs) queried their respective safety databases covering a 3-year period and provided aggregate ICSR submission statistics for expedited safety reports to an independent project manager. As measured in the US Food and Drug Administration (FDA)’s Adverse Event Reporting System (FAERS), ICSR volume for these seven MCs makes up approximately 20% of the total case volume. Aggregate metrics were calculated from the company data, specifically: (i) number of ICSR transmissions, (ii) average number of recipients (ANR) per case version transmitted, (iii) a submission selectivity metric, which measures the percentage of recipients not having received all sequential case version numbers, and (iv) percent of common ISCRs residing in two or more MAH databases. Results The analysis reflects 2,539,802 case versions, distributed through 7,602,678 submissions. The overall mean replication rate is 3.0 submissions per case version. The distribution of the ANR replication measure was observed to be very long-tailed, with a significant fraction of case versions (~ 12.4% of all transmissions) being sent to ten or more HA recipients. Replication is higher than average for serious, unlisted, and literature cases, ranging from 3.5 to 6.1 submissions per version. Within the subset of ICSR versions sent to three recipients, a significant degree of variability in the actual recipients (i.e., HAs) was observed, indicating that there is not one single combination of the same three HAs predominantly receiving an ICSR. Submission selectivity increases with the case version. For case version 6, the range of the submission selectivity for the MAHs ranges from ~ 10% to over 50%, with a median of 30.2%. Within the participating MAHs, the percentage of cases that reside within at least two safety databases is approximately 2% across five databases. Further analysis of the data from three MAHs showed percentages of 13.4%, 15.6%, and 27.9% of ICSRs originating from HAs and any other partners such as other MAHs and other institutions....

No abstract