Multiple Hypothesis Testing

Shaffer, Juliet Popper

doi:10.1146/annurev.ps.46.020195.003021

Cited by 1,516 publications

(566 citation statements)

References 0 publications

Supporting

Mentioning

553

Contrasting

Unclassified

Order By: Relevance

“…Besides the Bonferroni correction there are other alternative methods of FWER correction, like the Tukey Honestly Significant Difference test, the Scheffe test, Holm's method, Sidak's method; Hochberg's method, etc. Some of these corrections also take the dependency (non-independence) of tests into account (see e.g., Shaffer, 1995;Nichols and Hayasaka, 2003 for review).…”

Section: Family-wise Error Rate (Fwer) and Fdr Correction In Nhstmentioning

confidence: 99%

When null hypothesis significance testing is unsuitable for research: a reassessment

Szűcs

Ioannidis

2016

Preprint

107

View full text Add to dashboard Cite

Null hypothesis significance testing (NHST) has several shortcomings that are likely contributing factors behind the widely debated replication crisis of (cognitive) neuroscience, psychology, and biomedical science in general. We review these shortcomings and suggest that, after sustained negative experience, NHST should no longer be the default, dominant statistical practice of all biomedical and psychological research. If theoretical predictions are weak we should not rely on all or nothing hypothesis tests. Different inferential methods may be most suitable for different types of research questions. Whenever researchers use NHST they should justify its use, and publish pre-study power calculations and effect sizes, including negative findings. Hypothesis-testing studies should be pre-registered and optimally raw data published. The current statistics lite educational approach for students that has sustained the widespread, spurious use of NHST should be phased out.

show abstract

Section: Family-wise Error Rate (Fwer) and Fdr Correction In Nhstmentioning

confidence: 99%

When null hypothesis significance testing is unsuitable for research: a reassessment

Szűcs

Ioannidis

2016

Preprint

107

View full text Add to dashboard Cite

show abstract

“…Since we have n test statistics we need to correct for multiple testing [Schaffer, 1995] by calculating the family wise error rate, that is the probability of at least one false discovery under the global null that no loci are associated to disease. This is equivalent to combining all single locus tests into a single test statistic through their maximum value (minimum pvalue), giving…”

Section: Test Statisticsmentioning

confidence: 99%

Analysis of multiple SNPs in a candidate gene or region

Chapman

Whittaker

2008

Genetic Epidemiology

136

View full text Add to dashboard Cite

We consider the analysis of multiple SNPs within a gene or region. The simplest analysis of such data is based on a series of single SNP hypothesis tests, followed by correction for multiple testing, but it is intuitively plausible that a joint analysis of the SNPs will have higher power, particularly when the causal locus may not have been observed. However, standard tests, such as a likelihood ratio test based on an unrestricted alternative hypothesis, tend to have large numbers of degrees of freedom and hence low power. This has motivated a number of alternative test statistics. Here we compare several of the competing methods, including the multivariate score test (Hotelling's test) of Chapman et al. [2003], Fisher's method for combining p-values, the minimum p-value approach, a Fourier transform based approach recently suggested by Wang and Elston [2007] and a Bayesian score statistic proposed for microarray data by Goeman et al. [2005]. Some relationships between these methods are pointed out, and simulation results given to show that the minimum p-value and the Goeman et al. [2005] approaches work well over a range of scenarios. The Wang and Elston approach often performs poorly; we explain why, and show how its performance can be substantially improved.

show abstract

“…To determine whether the differences between conditions were significant we performed the paired samples t-test (in the logistic z domain; d.f. = 65) with a significance threshold of 0.01, applying Holm's procedure to control for multiple comparisons [Shaffer, 1995]. All differences were found to be significant with the exception of the following pairs:…”

Section: Resultsmentioning

confidence: 98%

Delayed Decision-making in Real-time Beatbox Percussion Classification

Stowell

Plumbley

2010

Journal of New Music Research

View full text Add to dashboard Cite

Real-time classification applied to a vocal percussion signal holds potential as an interface for live musical control. In this article we propose a novel approach to resolving the tension between the needs for low-latency reaction and reliable classification, by deferring the final classification decision until after a response has been initiated. We introduce a new dataset of annotated human beatbox recordings, and use it to study the optimal delay for classification accuracy. We then investigate the effect of such delayed decision-making on the quality of the audio output of a typical reactive system, via a MUSHRA-type listening test. Our results show that the effect depends on the output audio type: for popular dance/pop drum sounds the acceptable delay is on the order of 12-35 ms.

show abstract

Multiple Hypothesis Testing

Cited by 1,516 publications

References 0 publications

When null hypothesis significance testing is unsuitable for research: a reassessment

When null hypothesis significance testing is unsuitable for research: a reassessment

Analysis of multiple SNPs in a candidate gene or region

Delayed Decision-making in Real-time Beatbox Percussion Classification

Contact Info

Product

Resources

About