Multiplicity issues in clinical trials

Abstract

The three papers presented in this special issue on multiplicity issues in clinical trials are based on a workshop that was held at the Federal Institute for Drugs and Medical Devices (BfArM). The workshop was organized to discuss new methods and emerging issues related to multiple testing procedures between industry statisticians, regulators and academia.Multiple testing problems arise in the context of clinical trials from a couple of different sources. Multiple treatment comparisons due to, e.g. different dosages or regimens, multiple endpoints, multiple interim looks, multiple populations or even combinations of these can cause an inflation of the familywise type 1 error. The consequence is an increased probability of wrong decisions in the contexts of drug approval and reimbursement if the analysis strategy does not account properly for multiplicity.In 2002, the EMA Points to Consider on Multiplicity Issues in Clinical Trials came into operation (EMEA CPMP, 2002). Since then, it has been proven to be useful for both, industry and regulators when planning and assessing confirmatory clinical trials. Meanwhile, however, methodological advances have been made in more complex multiplicity settings resulting in more powerful and flexible procedures in situations where several hypotheses are to be tested, eventually with different degrees of prominence attributed to the individual questions. In line with the development of these methods there is an increasing complexity of the primary and secondary hypothesis framework in confirmatory clinical trials. On the one hand, the applicant wishes to answer a number of questions within a single trial that potentially help to improve and accelerate clinical development. On the other hand, it may be in regulator's interest to ensure statistical evidence in different claims to be made with a clinical trial.Some powerful methods have recently been developed, e.g. the different gatekeeping and fallback procedure as well as elegant graphical solutions. The increasing complexity of hypothesis frameworks and methods, however, results in new issues and poses questions on general principles that have not been considered so far. These include consistency problems or the construction of simultaneous confidence intervals.While the consideration of multiplicity issues in confirmatory clinical trials represents an accepted international standard, there is a persisting discussion about the relevance of multiple comparison procedures in other areas of medical research such as epidemiology (Rothman, 1990;Bender and Lange, 2001). In systematic reviews and meta-analyses the general problem of multiplicity has been more or less neglected for decades. However, the recent work of the Statistical Methods Group of Cochrane Collaboration and the Institute for Quality and Efficiency in Health Care led to the inclusion of key recommendations regarding the multiplicity problem in the new version of the Cochrane Handbook (Higgins et al., 2008;Bender, 2009).

A Vector Theory of Assessing Clinical Trials: An Application to Bioequivalence

“…Another important aspect in clinical trials with multiple endpoints is multiplicity, which refers to the increased risk of false positives (i.e., type I errors) when multiple statistical tests are conducted simultaneously or sequentially within a study or analysis [18][19][20][21][22]. The issue of multiplicity is widely acknowledged, and suitable methods are commonly used to avoid it; however, until now, no measures have been taken to address endpoints that are related to one another, which is the focus of this study.…”

“…Multiplicity, which refers to the inclusion of multiple endpoints in clinical trials, can present several challenges such as the increased risk of false-positive findings (type I error) since conducting multiple statistical tests raises the probability of observing significant results by chance alone [15][16][17]; however, while avoiding multiplicity in clinical trials is important to minimize the risk of false-positive findings, it is also important to avoid the risk of increasing type II error (false negatives) [18][19][20][21][22]. The latter can arise from several factors like small sample size, inadequate study design, and insufficient statistical power.…”