A Vast Graveyard of Undead Theories

Ferguson, Christopher J.; Heene, Moritz

doi:10.1177/1745691612459059

Cited by 505 publications

(253 citation statements)

References 57 publications

(96 reference statements)

Supporting

Mentioning

248

Contrasting

Unclassified

Order By: Relevance

“…In true experiments, top-tier journals and second-tier journals/book chapters showed moderate effects and did not differ significantly from one another. However, the three unpublished dissertations showed negligible effects, which is consistent with reports of bias against the publication of nonsignificant findings (Ferguson & Heene, 2012;Scherer, Dickersin, & Langenberg, 1994). While we included several unpublished studies in each meta-analysis, all of these were dissertations accessed through Dissertation Abstracts International.…”

Section: Implications Of Moderators Of Video-game Effectssupporting

confidence: 84%

Effects of video-game play on information processing: A meta-analytic investigation

et al. 2013

View full text Add to dashboard Cite

Do video games enhance cognitive functioning? We conducted two meta-analyses based on different research designs to investigate how video games impact informationprocessing skills (auditory processing, executive functions, motor skills, spatial imagery, and visual processing). . Whereas the quasi-experimental studies yielded small to large effect sizes across domains, the true experiments yielded negligible effects for executive functions, which contrasted with the small to medium effect sizes in other domains. The quasi-experimental studies appeared more susceptible to bias than were the true experiments, with larger effects being reported in higher-tier than in lower-tier journals, and larger effects reported by the most active research groups in comparison with other labs. The results are further discussed with respect to other moderators and limitations in the extant literature.

show abstract

Section: Implications Of Moderators Of Video-game Effectssupporting

confidence: 84%

Effects of video-game play on information processing: A meta-analytic investigation

et al. 2013

View full text Add to dashboard Cite

show abstract

“…Once such exploratory effects are detectedregardless of whether they are actually false positives -they are easily transformed into post-hoc hypotheses given the strength of the hindsight bias that allows one to always find a plausible rationale for almost any hypothesis (Fischhoff, 1975). In the worst-case scenario, HARKing causes a false positive to become a new (yet wrong from the outset) theory, which ultimately undermines the quality of produced science (see Ferguson & Heene, 2012, for a review). Given the relative high prevalence of HARKing among researchers (John et al, 2012;Kerr, 1998), it is likely that many supposedly "unsuccessful" studies were actually devoted to replicating invalid, yet presented as confirmatory, effects.…”

Section: Questionable Research Practices Cause Questionable Research mentioning

confidence: 99%

“…This phenomenon is widely known as the "file drawer" problem (Fanelli, 2012;Ferguson & Heene, 2012;Rosenthal, 1979): Positive, statistically significant results get more easily published than negative, statistically non-significant results. To provide with an extreme, hypothetical version of this problem, Rosenthal somewhat cynically stated that "the journals are filled with the 5% of the studies that show Type I errors, while the file drawers back at the lab are filled with the 95% of the studies that show non-significant (e.g., p > 0.05) results".…”

Section: Publication Standards Promote Bad Research Practicesmentioning

confidence: 99%

“…A non-significant p-value provides neither evidence for nor against the null hypothesis, and even when the latter is true, the p-value can take almost any value ( Cumming, 2012;Dienes, 2011). 4 Thus, even when a theory lacks empirical support and when failed attempts at replication accumulate, it is still hardly possible to properly operate the falsification mechanism, and this without even mentioning the fact that publication bias may prevent failed replications to be published (Ferguson & Heene, 2012).…”

Section: Bringing Social Psychology Back To Its Name: Toward a Perspementioning

confidence: 99%

See 1 more Smart Citation

Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future

Świątkowski¹,

Dompnier

2017

rips

View full text Add to dashboard Cite

Over the last few years, psychology researchers have become increasingly preoccupied with the question of whether findings from psychological studies are generally replicable. The debates have originated from some unfortunate events of scientific misconduct in the field, and they have reached a climax with the recent discovery of a relatively weak rate of replicability of published literature, leading to the so-called replicability crisis in psychology.The present paper is concerned with examining the issue of replicability in the field of social psychology. We begin by drawing a state of the art of the crisis in this field. We then highlight some possible causes for the crisis, discussing topics of statistical power, questionable research practices, publication standards, and hidden auxiliary assumptions of context-dependency of social psychological theories. Finally, we argue that given the absence of absolute falsification in science, social psychology could greatly benefit from adopting McGuire's perspectivist approach to knowledge construction.

show abstract

“…It has been argued that replication may uncover false positives (e.g., Diekmann, 2011;Open Science Collaboration, 2012;Simmons et al, 2011), but this only holds if studies with nonsignificant results are accessible to researchers (see also Ferguson & Heene, 2012).…”

Section: General Implicationsmentioning

confidence: 99%

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Nuijten¹

2018

Preprint

View full text Add to dashboard Cite

SUMMARY DOCTORAL DISSERTATION: Psychology is facing a “replication crisis”. Many psychological findings could not be replicated in novel samples, which lead to the growing concern that many published findings are overly optimistic or even false. In this dissertation, we investigated potential indicators of problems in the published psychological literature. In Part I of this dissertation, we looked at inconsistencies in reported statistical results in published psychology papers. To facilitate our research, we developed the free tool statcheck; a “spellchecker” for statistics. In Part II, we investigated bias in published effect sizes. We showed that in the presence of publication bias, the overestimation of effects can become worse if you combine studies. Indeed, in meta-analyses from the social sciences we found strong evidence that published effects are overestimated. These are worrying findings, and it is important to think about concrete solutions to improve the quality of psychological research. Some of the solutions we propose are preregistration, replication, and transparency. We argue that to select the best strategies to improve psychological science, we need research on research: meta-research.

show abstract

A Vast Graveyard of Undead Theories

Cited by 505 publications

References 57 publications

Effects of video-game play on information processing: A meta-analytic investigation

Effects of video-game play on information processing: A meta-analytic investigation

Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future

Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological Science

Contact Info

Product

Resources

About