Signal detection using change point analysis in postmarket surveillance

Abstract

PurposeSignal detection methods have been used extensively in postmarket surveillance to identify elevated risks of adverse events associated with medical products (drugs, vaccines, and devices). However, current popular disproportionality methods ignore useful information such as trends when the data are aggregated over time for signal detection.MethodsIn this paper, we applied change point analysis (CPA) to trend analysis of medical products in a spontaneous adverse event reporting system. CPA was used to detect the time point at which statistical properties of a sequence of observations change over time. Two CPA approaches, change in mean and change in variance, were demonstrated by an example using neurostimulator adverse event dataset.ResultsTwo significant change points associated with upward trends were detected in June 2008 (n = 20, p < 0.001) and May 2011 (n = 51, p = 0.003). Further investigation confirmed battery issues and expansion of the indication for use could be possible causes for the occurrence of these change points. Two time points showed extremely low number of loss of therapy events, two cases in October 2009 and three in November 2009, which could be the result of reporting issues such as underreporting.ConclusionAs a complimentary tool to current signal detection efforts at FDA, CPA can be used to detect changes in the association between medical products and adverse events over time. Detecting these changes could be critical for public health regulation, adverse events surveillance, product recalls, and regulators’ understanding of the connection between adverse events and other events regarding regulated products. © 2015 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.

Benefits of combining change‐point analysis with disproportionality analysis in pharmacovigilance signal detection

“…The same results were obtained with data from BNPV. Z. Xu et al found that change points in the count of report may be related to some issues like extended indication for use for devices. Similarly, we found that the increase in the number of reports related to benfluorex and ADRs may be due to the marketing suspension of this drug in November 2009.…”

“…However, compared with change in mean, this approach is less sensitive to detect changes in safety data. 6 In the method elaboration phase, the CPA method was applied on the count of reports per month and the monthly PRR-. We defined trends of segments between detected change points, especially trends of measures after the moment that each couple was defined as a SDR.…”

“…Besides the change in mean approach, there is another approach called change in variance which can be used to perform CPA. However, compared with change in mean, this approach is less sensitive to detect changes in safety data …”

“…We defined trends of segments between detected change points, especially trends of measures after the moment that each couple was defined as a SDR. We compared results obtained for PRR‐ with the monthly count of reports which was previously investigated by Z. Xu et al In the method validation phase, CPA was only applied for the chosen measure.…”

“…In the field of public health surveillance, CPA was applied for the first time by Kass‐Hout et al to the active syndromic surveillance data to detect changes in the incidence of emergency department visits due to daily influenza like illness during the H1N1 pandemic. This method was also demonstrated by Z. Xu et al in a study using the Manufacturer and User Facility Device Experience (MAUDE) database coordinated by the United State Food and Drug Administration (FDA) that it can be applied to detect changes in the association between medical devices and adverse drug reaction overtime and can be useful for signal detection.…”