Confidence intervals and replication: Where will the next mean fall?

Cumming, Geoff; Maillardet, Robert

doi:10.1037/1082-989x.11.3.217

Cited by 119 publications

(101 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cumming and Maillardet (2006) stated that "considering whether an effect is replicable is at the heart of drawing inferences from data" (p. 217). Furthermore, although the additivity of the Stroop effect with SOA observed by Fagot and Pashler (1992, Experiment 7) plays a crucial role in the theoretical argumentation of Dell'Acqua et al (2007), there are no reported replications of this additivity in the literature.…”

Section: The Present Studymentioning

confidence: 99%

Locus of semantic interference in picture naming: Evidence from dual-task performance.

Piai¹,

Roelofs²,

Schriefers³

2014

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

Disagreement exists regarding the functional locus of semantic interference of distractor words in picture naming. This effect is a cornerstone of modern psycholinguistic models of word production, which assume that it arises in lexical response-selection. However, recent evidence from studies of dual-task performance suggests a locus in perceptual or conceptual processing, prior to lexical response-selection. In these studies, participants manually responded to a tone and named a picture while ignoring a written distractor word. The stimulus onset asynchrony (SOA) between tone and picture-word stimulus was manipulated. Semantic interference in naming latencies was present at long tone pre-exposure SOAs, but reduced or absent at short SOAs. Under the prevailing structural or strategic response-selection bottleneck and central capacity sharing models of dual-task performance, the underadditivity of the effects of SOA and stimulus type suggests that semantic interference emerges before lexical response-selection. However, in more recent studies, additive effects of SOA and stimulus type were obtained. Here, we examined the discrepancy in results between these studies in 6 experiments in which we systematically manipulated various dimensions on which these earlier studies differed, including tasks, materials, stimulus types, and SOAs. In all our experiments, additive effects of SOA and stimulus type on naming latencies were obtained. These results strongly suggest that the semantic interference effect arises after perceptual and conceptual processing, during lexical response-selection or later. We discuss several theoretical alternatives with respect to their potential to account for the discrepancy between the present results and other studies showing underadditivity. Keywords: dual-task performance, picture-word interference, response-selection bottleneck, semantic interference, Stroop An important question in the psychology of language concerns how speakers select from memory the words that they want to produce. This ability, called lexical selection, is a topic of much research in the field of word production. One way of studying lexical selection consists of presenting participants with pictured objects paired with superimposed distractor words, a paradigm called picture-word interference (PWI; see Abdel Rahman & Melinger, 2009;Glaser, 1992;Roelofs, 2007, for reviews). Participants are instructed to name the pictures and to ignore the distractors. The relation the distractor word bears with the picture name (e.g., semantic, phonological, etc.) is manipulated and effects obtained are thought to inform researchers about processes involved in word production.One specific effect has long been assumed to provide evidence about the nature of lexical selection: semantic interference (e.g., Damian & Martin, 1999;Levelt, Roelofs, & Meyer, 1999;Roelofs, 1992;Schriefers, Meyer, & Levelt, 1990;Starreveld & La Heij, 1996). This effect concerns the finding that response times (RTs) are longer for picture naming when the dis...

show abstract

Section: The Present Studymentioning

confidence: 99%

Locus of semantic interference in picture naming: Evidence from dual-task performance.

Piai¹,

Roelofs²,

Schriefers³

2014

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

show abstract

“…In 20 of the 24 cases, the 95% CI includes the mean next above it in the figure. That is 83.3% of the experiments, which happens to be very close to the longrun average of 83.4% (Cumming & Maillardet, 2006;Cumming, Williams, & Fidler, 2004). So a 95% CI is an 83% prediction interval for the ES estimate of a replication experiment.…”

Section: Replication P Values and Cismentioning

confidence: 99%

The New Statistics

2013

Self Cite

View full text Add to dashboard Cite

We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research literature is incomplete and untrustworthy, we need new requirements to ensure research integrity. These include prespecification of studies whenever possible, avoidance of selection and other inappropriate dataanalytic practices, complete reporting, and encouragement of replication. Second, in response to renewed recognition of the severe flaws of null-hypothesis significance testing (NHST), we need to shift from reliance on NHST to estimation and other preferred techniques. The new statistics refers to recommended practices, including estimation based on effect sizes, confidence intervals, and meta-analysis. The techniques are not new, but adopting them widely would be new for many researchers, as well as highly beneficial. This article explains why the new statistics are important and offers guidance for their use. It describes an eight-step new-statistics strategy for research with integrity, which starts with formulation of research questions in estimation terms, has no place for NHST, and is aimed at building a cumulative quantitative discipline.Keywords research integrity, the new statistics, estimation, meta-analysis, replication, statistical analysis, research methods Received 7/8/13; Revision accepted 8/20/13 8 Cumming sufficient guidance is available for us to make the changes now, even as we develop further new-statistics practices. The changes will prove highly worthwhile: Our publicly available literature will become more trustworthy, our discipline more quantitative, and our research progress more rapid.

show abstract

“…Because of the importance of replication probability and the confusion surrounding it, recent articles in numerous disciplines have urged researchers to consider replication probability more carefully (e.g., Cumming, 2008;Cumming & Maillardet, 2006;Gorroochurn, Hodge, Heiman, Durner, & Greenberg, 2007;Greenwald, Gonzalez, Harris, & Guthrie, 1996;Killeen, 2005;Robinson & Levin, 1997;Sohn, 1998). Researchers have been offered formulas with which to compute the probability of replicating their current results, and they have been advised to report the resulting replication probabilities as well as-or even in preference to-more traditional statistical measures (e.g., Greenwald et al, 1996;Killeen, 2005;Psychological Science editorial board, 2005).…”

mentioning

confidence: 99%

What is the probability of replicating a statistically significant effect?

Miller

2009

Psychonomic Bulletin & Review

View full text Add to dashboard Cite

Confidence intervals and replication: Where will the next mean fall?

Cited by 119 publications

References 21 publications

Locus of semantic interference in picture naming: Evidence from dual-task performance.

Locus of semantic interference in picture naming: Evidence from dual-task performance.

The New Statistics

What is the probability of replicating a statistically significant effect?

Contact Info

Product

Resources

About