A systematic review of substandard, falsified, unlicensed and unregistered medicine sampling studies: a focus on context, prevalence, and quality

Abstract

Substandard and falsified (SF) medicines are a global issue contributing to antimicrobial resistance and causing economic and humanitarian harm. To direct law enforcement efficiently, halt the spread of SF medicines and antimicrobial resistance, academics, NGOs and government organisations use medicine quality sampling studies to estimate the prevalence of the problem. A systematic review of medicine quality studies was conducted to estimate how the methodological quality of these studies and SF prevalence has changed between 2013 and 2018. We also aimed to critique medicine sampling study methodologies, and the systematic review process which generates prevalence estimates. Based on 33 studies, the overall estimated median (Q1–Q3) prevalence of SF medicines appears to have remained high at 25% (7.7%–34%) compared with 28.5% in 2013. Furthermore, the methodological quality of prevalence studies has improved over the last 25 years. Definitive conclusions regarding the prevalence of SF medicines cannot be drawn due to the variability in sample sizes, consistency of design methods, and a lack of information concerning contextual factors affecting medicine quality studies. We contend that studies which present cumulative average prevalence figures are useful in a broad sense but could be improved to create more reliable estimates. We propose that medicine quality studies record the context of the study environment to allow systematic reviewers to compare like with like. Although, the academic rigour of medicine quality studies is improving, medicine sampling study limitations still exist. These limitations inhibit the accurate estimation of SF medicine prevalence which is needed to support detailed policy changes.

Identification of Substandard and Falsified Medicines: Influence of Different Tolerance Limits and Use of Authenticity Inquiries

“…As correctly stated by McManus and Naughton, 5 the results of systematic reviews are only as reliable as the original data to which they refer. If the arbitrary choice of tolerance limits allows the rate of OOS medicines in the same study to vary by a factor of more than 10 (as shown in Figure 2 ), the results of systematic reviews obviously become unreliable, and, for example, comparisons of the rates of OOS medicines between different time periods, as attempted in the systematic review by McManus and Naughton, 5 may become meaningless.…”

“…As correctly stated by McManus and Naughton, 5 the results of systematic reviews are only as reliable as the original data to which they refer. If the arbitrary choice of tolerance limits allows the rate of OOS medicines in the same study to vary by a factor of more than 10 (as shown in Figure 2 ), the results of systematic reviews obviously become unreliable, and, for example, comparisons of the rates of OOS medicines between different time periods, as attempted in the systematic review by McManus and Naughton, 5 may become meaningless. Given the important public health and socioeconomic effects of SF medicines, and the importance of reliable and comparable prevalence data for such medicines, a harmonization of the tolerance limits used to distinguish compliant from OOS products in different medicine quality studies is urgently called for, as also concluded from a recent study by the Brazilian Health Regulatory Agency.…”

“…Notably, these different tolerance limits have been used in recent, large medicine quality studies, 7 , 17 , 28 and the results of these studies have been included into recent systematic reviews, which then calculated overall prevalence rates of SF medicines by aggregating numbers of “poor-quality” medicines reported in the included studies. 3 , 5 …”

“…Similarly, a systematic review of 96 studies up to the year 2017 (total 67,839 samples) by Ozawa et al 3 reported a prevalence of 13.6%. By contrast, a systematic review of 15 studies up to January 2013 (total 3,931 samples) by Almuazini et al 4 reported a prevalence of 28.5%, and a consecutive review by McManus and Naughton, 5 using the same methodology as Almuazini et al 4 and including 33 studies published from January 2013 until December 2018 (total 19,921 samples), reported a prevalence of 25%. Notably, none of these four reviews was able to state the rate of falsified medicines separately from the rate of substandard medicines because the heterogeneity of the methodologies and definitions used in the reviewed studies did not allow to aggregate prevalence data for falsified medicines.…”

“…Guidelines for the conduct and the reporting of medicine quality studies have been published by Newton et al 13 and by the WHO, 14 and as documented by McManus and Naughton, 5 this led to a noticeable improvement of the quality of such studies in the recent years.…”