The Replication Recipe: What makes for a convincing replication?

Brandt, Mark J.; IJzerman, Hans; Dijksterhuis, Ap; Farach, Frank J.; Geller, Jason; Giner‐Sorolla, Roger; Grange, James A.; Perugini, Marco; Spies, Jeffrey R.; Veer, Anna Elisabeth van 't

doi:10.1016/j.jesp.2013.10.005

Cited by 600 publications

(466 citation statements)

References 39 publications

Supporting

Mentioning

431

Contrasting

Unclassified

Order By: Relevance

“…As Brandt et al (2014) pointed out, a replication in psychological research will never be absolutely exact or direct (see also, Stroebe & Strack, 2014), which is, of course, also the case in the present research. Major differences compared to the original study are: 1) In contrast to Elliot et al who tested each participant individually, we tested the whole sample at once in a group session; 2) we tested German participants instead of a sample from the United States (note, however, that Experiments 5a and 5b of Elliot et al's original series were conducted in Germany as well), which also required using German translations of Elliot et al's original questions for the assessment of perceived attractiveness; and 3) we utilized a larger sample size (total N = 89, female subsample n = 72), which yielded a high test power (total sample 0.993, female subsample 0.978).…”

Section: Discussionsupporting

confidence: 47%

Men in red: A reexamination of the red-attractiveness effect

2015

View full text Add to dashboard Cite

Elliot, Kayser, Greitemeyer, Lichtenfeld, Gramzow, Maier, and Liu (Journal of Experimental Psychology: General, 139(3), [399][400][401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417] 2010) showed that presenting men in front of a red background or with a red shirt enhances their attractiveness, sexual desirability, and status in the eyes of female observers. The purpose of the present research was to gain further insights concerning the robustness and the ecological validity of this red effect. In two experiments, we replicated the basic paradigm used by Elliot et al. Experiment 1 was a close replication of the first experiment in their original series. We presented the photo of a young man used by Elliot et al. on either a red or white background and asked participants (N = 89, female subsample n = 72) to rate it with regard to perceived attractiveness. Experiment 2 (N = 32) represents a somewhat more complex version of the first experiment; we increased the variance of the stimuli by showing photos of multiple men wearing different apparel styles (formal and casual, respectively). We did not find any significant impact of red in either of the studies. What we found, however, was a significant effect of apparel style with attractiveness ratings being higher for men wearing formal apparel than for men wearing casual apparel. Our results question the robustness and the ecological validity of Elliot et al.'s finding. On a more general level, they further point to limitations arising from (often necessary) restrictions in experimental designs.Keywords Replication . Color . Attractiveness . Red effect . Status Elliot et al. (2010) presented a series of experiments suggesting that the color red enhances male attractiveness and sexual desirability as perceived by women. They further presented evidence suggesting that this effect is mediated by an increase in perceived status. Basically, all seven experiments reported by Elliot et al. followed the same between-subjects design. Participants viewed the photo of a male target person whom they had to judge with regard to different variables, such as "perceived attractiveness," "sexual attraction," and "perceived status" (original variable labels used by Elliot et al.). The color of the background (Experiments 1-3) or the depicted man's shirt (Experiments 4-7), respectively, was manipulated as being either red or a contrasting color, e.g., white (see Table 1 for further details on the specific experiments). An enhancing effect of red was found in five of five experiments for perceived attractiveness and in three of three experiments for sexual attraction (note: in some experiments, more than one dependent variable was used). The mediation of the red effect via an increase in status could be further demonstrated in all three experiments where this variable was addressed via two different approaches: 1) the experimental-causal-chain approach, and 2) the measurement-of-mediation approach.The accumulated results of Elliot et al. (2010)...

show abstract

Section: Discussionsupporting

confidence: 47%

Men in red: A reexamination of the red-attractiveness effect

2015

View full text Add to dashboard Cite

show abstract

“…In the light of recent calls for high-powered replication studies (see, e.g., Brandt et al, 2014), we encounter more and more situations in which the replication study is actually larger than the original study. In those cases, the combined effect size estimate will have less bias than the effect size estimate of just the smaller, original study.…”

Section: General Implicationsmentioning

confidence: 99%

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

SUMMARY DOCTORAL DISSERTATION: Psychology is facing a “replication crisis”. Many psychological findings could not be replicated in novel samples, which lead to the growing concern that many published findings are overly optimistic or even false. In this dissertation, we investigated potential indicators of problems in the published psychological literature. In Part I of this dissertation, we looked at inconsistencies in reported statistical results in published psychology papers. To facilitate our research, we developed the free tool statcheck; a “spellchecker” for statistics. In Part II, we investigated bias in published effect sizes. We showed that in the presence of publication bias, the overestimation of effects can become worse if you combine studies. Indeed, in meta-analyses from the social sciences we found strong evidence that published effects are overestimated. These are worrying findings, and it is important to think about concrete solutions to improve the quality of psychological research. Some of the solutions we propose are preregistration, replication, and transparency. We argue that to select the best strategies to improve psychological science, we need research on research: meta-research.

show abstract

“…Replication is an essential aspect of any research programme interested in building a cumulative science (Brandt et al, 2014), and replicability of empirical results in psychological science is starting to gain interest in the experimental community (Open Science Collaboration, 2012) In this paper, we have conducted two relatively close replications of extant work (Experiment 1 was a close replication of Horoufchin et al, 2011a, and Experiment 3 was a close replication of Meiran et al, 2000). In both cases, we report different results to those reported by the original authors.…”

Section: On the Replicability Of Rci Effectsmentioning

confidence: 99%

Can time-based decay explain temporal distinctiveness effects in task switching?

Grange

Cross

2015

Quarterly Journal of Experimental Psychology

Self Cite

View full text Add to dashboard Cite

In task switching, extending the response-cue interval (RCI) reduces the switch cost-the detriment to performance when switching compared to repeating tasks. This reduction has been used as evidence for the existence of task-set decay processes. Recently, this has been challenged by the observation of sequential dependencies on the RCI effect: switch cost is only reduced at longer RCIs when the previous trial had a short RCI. This trialwise variation of RCI is thought to affect the temporal distinctiveness (TD) of a previous task's episodic trace, affecting the probability of its automatic retrieval on the current trial; importantly, TD is thought to be independent of the current trial's RCI. The present study highlights a dependency between the current RCI and TD, and demonstrates that a decay model can reproduce some patterns of data attributed to TD. Further, the decay account makes a strong prediction when TD is held constant: repetition response times should slow as RCI increases, and switch response times should be facilitated. This prediction was tested via re-analysis of extant data and three experiments. The re-analysis provided some evidence for the decay account, but Experiments 1 and 2 report slowing for task repetition and switch trials, which cannot be explained by a task-set decay process. Experiment 3, which utilised tasks requiring perceptual judgements, showed small evidence for decay. We conclude that the data are largely consistent with the TD account and that the evidence for decay of higher-level task-sets is not convincing.

show abstract

The Replication Recipe: What makes for a convincing replication?

Cited by 600 publications

References 39 publications

Men in red: A reexamination of the red-attractiveness effect

Men in red: A reexamination of the red-attractiveness effect

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Can time-based decay explain temporal distinctiveness effects in task switching?

Contact Info

Product

Resources

About