A note on confidence bounds after fixed-sequence multiple tests

Tu, Yi Hsuan; Cheng, Bin; Cheung, Ying Kuen

doi:10.1016/j.jspi.2012.05.002

Cited by 3 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative method (the fixed sequence procedure) is to specify the sequence of the hypothesis testing (primary outcome, first secondary, second secondary, etc.) [ 53 ]. Then each test will be done at the chosen level of significance in the specified order (here both the confidence interval and the P-value can be used to demonstrate the threshold), but as soon as a test is non-significant then the remaining null hypotheses are accepted.…”

Section: Methods and Resultsmentioning

confidence: 99%

The thresholds for statistical and clinical significance – a five-step procedure for evaluation of intervention effects in randomised clinical trials

Jakobsen

Gluud

Winkel

et al. 2014

BMC Med Res Methodol

164

190

View full text Add to dashboard Cite

BackgroundThresholds for statistical significance are insufficiently demonstrated by 95% confidence intervals or P-values when assessing results from randomised clinical trials. First, a P-value only shows the probability of getting a result assuming that the null hypothesis is true and does not reflect the probability of getting a result assuming an alternative hypothesis to the null hypothesis is true. Second, a confidence interval or a P-value showing significance may be caused by multiplicity. Third, statistical significance does not necessarily result in clinical significance. Therefore, assessment of intervention effects in randomised clinical trials deserves more rigour in order to become more valid.MethodsSeveral methodologies for assessing the statistical and clinical significance of intervention effects in randomised clinical trials were considered. Balancing simplicity and comprehensiveness, a simple five-step procedure was developed.ResultsFor a more valid assessment of results from a randomised clinical trial we propose the following five-steps: (1) report the confidence intervals and the exact P-values; (2) report Bayes factor for the primary outcome, being the ratio of the probability that a given trial result is compatible with a ‘null’ effect (corresponding to the P-value) divided by the probability that the trial result is compatible with the intervention effect hypothesised in the sample size calculation; (3) adjust the confidence intervals and the statistical significance threshold if the trial is stopped early or if interim analyses have been conducted; (4) adjust the confidence intervals and the P-values for multiplicity due to number of outcome comparisons; and (5) assess clinical significance of the trial results.ConclusionsIf the proposed five-step procedure is followed, this may increase the validity of assessments of intervention effects in randomised clinical trials.

show abstract

Section: Methods and Resultsmentioning

confidence: 99%

The thresholds for statistical and clinical significance – a five-step procedure for evaluation of intervention effects in randomised clinical trials

Jakobsen

Gluud

Winkel

et al. 2014

BMC Med Res Methodol

164

190

View full text Add to dashboard Cite

show abstract

“…In addition, Hommel and Kropf introduced a specific fixed sequence procedure, which allows a pre‐specified number k of acceptances and has the same critical value α / k . For a detailed review of recent developments in this area of research, see Dmitrienko et al and Wiens and Dmitrienko , and for applications of fixed sequence multiple testing procedures in different fields, see Alosh and Huque and Tu et al .…”

Section: Introductionmentioning

confidence: 99%

On generalized fixed sequence procedures for controlling the FWER

Qiu

Lynch

2015

Statistics in Medicine

View full text Add to dashboard Cite

Testing a sequence of pre-ordered hypotheses to decide which of these can be rejected or accepted while controlling the familywise error rate (FWER) is of importance in many scientific studies such as clinical trials. In this paper, we first introduce a generalized fixed sequence procedure whose critical values are defined by using a function of the numbers of rejections and acceptances, and which allows follow-up hypotheses to be tested even if some earlier hypotheses are not rejected. We then construct the least favorable configuration for this generalized fixed sequence procedure and present a sufficient condition for the FWER control under arbitrary dependence. Based on the condition, we develop three new generalized fixed sequence procedures controlling the FWER under arbitrary dependence. We also prove that each generalized fixed sequence procedure can be described as a specific closed testing procedure. Through simulation studies and a clinical trial example, we compare the power performance of these proposed procedures with those of the existing FWER controlling procedures. Finally, when the pairwise joint distributions of the true null p-values are known, we further improve these procedures by incorporating pairwise correlation information while maintaining the control of the FWER. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

“…The basic idea of our approach is that we do not shift the whole level of a rejected hypothesis to the next one, but keep some of the level for the construction of a positive confidence bound. A similar idea of error splitting for a fixed-sequence procedure in dose-finding studies was proposed by [14]. In order to loose not too much power for the following hypothesis, we specify the amount of reserved level in dependence of the observation.…”

Section: Introductionmentioning

confidence: 99%