Background: While traditional signal detection methods in pharmacovigilance are based on spontaneous reports, the use of social media is emerging. The potential strength of webbased data relies on their volume and real time availability allowing early signals of disproportionate reporting (SDRs) detection.Objectives: This study aimed at assessing (1) the consistency of SDRs detected from patient's medical forums in France compared to traditional reporting systems and (2) the ability of identifying SDRs earlier than the traditional reporting systems.
Methods:Messages posted on patients' forums between 2005 and 2015 were used. Eight disproportionality definitions were retained. Comparison of SDRs from the forums to SDRs detected in VigiBase® was done by describing the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, the Receiver Operating Characteristics curve and the area under the curve (AUC). Time difference in months between the detection dates of SDRs from the forums and from VigiBase® was provided.
Results:Comparison analysis showed that the sensitivity ranged from 29 to 50.6%, the specificity from 86.1 to 95.5%, the PPV from 51.2 to 75.4%, the NPV from 68.5 to 91.6% and the accuracy from 68 to 87.7%. The AUC reached 0.85 when using the metric EBGM. Up to 37.5% of the SDRs were detected earlier in the forums than in VigiBase®.
Conclusion:The specificity, PPV and NPV were high. The overall performance was good showing that data from medical forums may be a valuable source for signal detection. In total, up to 37.5% of the SDRs could have been detected earlier, thus facilitating increased safety of the patients. Further enhancements are needed to investigate the reliability and validation of patients' medical forums worldwide, the extension of this analysis to all possible drugs or at least to a wider selection of drugs as well as to further assess performance against established signals.