Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability

Nosek, Brian A.; Spies, Jeffrey R.; Motyl, Matt

doi:10.31219/osf.io/k4bgq

Cited by 604 publications

(843 citation statements)

References 100 publications

(145 reference statements)

Supporting

Mentioning

814

Contrasting

Unclassified

Order By: Relevance

“…The psychological sciences are seeing today a revival of concern in the robustness of our results, and particularly to what extent published results are indeed replicable. The problem likely emerges from the fact that the current reward scheme pushes researchers, reviewers, and editors to value more results where there is a p-value below the .05 threshold than results that are not significant (see e.g., Ioannidis, 2005;Nosek, Spies, & Motyl, 2012 for the general argument, and Open Science Collaboration, 2015 for a recent set of results in psychology). As a consequence, there is an over-representation of significant results in the literature, and quite likely an increase in the number of false positives that are present in published studies (Sterling, Rosenbaum, & Weinkam, 1995).…”

Section: Why Carry Out a Meta-analysis?mentioning

confidence: 99%

Can infants learn phonology in the lab? A meta-analytic answer

Cristià

2018

Cognition

View full text Add to dashboard Cite

Two of the key tasks facing the language-learning infant lie at the level of phonology: establishing which sounds are contrastive in the native inventory, and determining what their possible syllabic positions and permissible combinations (phonotactics) are. In 2002-2003, two theoretical proposals, one bearing on how infants can learn sounds (Maye, Werker, & Gerken, 2002) and the other on phonotactics (K. E. Chambers, Onishi, & Fisher, 2003), were put forward on the pages of Cognition, each supported by two laboratory experiments, wherein a group of infants was briefly exposed to a set of pseudo-words, and plausible phonological generalizations were tested in a subsequent phase. These two papers have received considerable attention from the general scientific community, and inspired a flurry of follow-up work. In the context of questions regarding the replicability of psychological science, the present work uses a meta-analytic approach to appraise extant empirical evidence for infant phonological learning in the laboratory. It is found that neither seminal finding (on learning sounds and learning phonotactics) holds up when close methodological replications are integrated, although less close methodological replications do provide some evidence in favor of the sound learning strand of work. Implications for authors and readers of this literature are drawn out. It would be desirable that additional mechanisms for phonological learning be explored, and that future infant laboratory work should employ paradigms that rely on constrained and unambiguous links between experimental exposure and measured infant behavior.

show abstract

Section: Why Carry Out a Meta-analysis?mentioning

confidence: 99%

Can infants learn phonology in the lab? A meta-analytic answer

Cristià

2018

Cognition

View full text Add to dashboard Cite

show abstract

“…A simple answer is that there are pragmatic barriers to sharing and few incentives to overcome them [15]. The present academic culture emphasizes publications and grants as researchers' primary incentives.…”

Section: Introductionmentioning

confidence: 99%

Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency

Kidwell¹,

Lazarević²,

Baranski³

et al. 2016

Preprint

Self Cite

135

167

View full text Add to dashboard Cite

Data Availability Statement: All data and materials are publicly accessible at https://osf.io/rfgdw/. Also, we preregistered our study design and analysis plan. You can find the preregistration at https://osf.io/ipkea/.Funding: The authors received no specific funding for this work.Competing Interests: Brian Nosek created the badges to acknowledge open practices, and Brian Nosek and Mallory Kidwell are on a committee maintaining the badges. The badges and specifications for earning them are CC0 licensed with and in four comparison journals. We report an increase in reported data sharing of more than an order of magnitude from baseline in Psychological Science, as well as an increase in reported materials sharing, although to a weaker degree. Moreover, we show that reportedly available data and materials were more accessible, correct, usable, and complete when badges were earned. We demonstrate that badges are effective incentives that improve the openness, accessibility, and persistence of data and materials that underlie scientific research.

show abstract

“…We also recognize that the sample size for Study 1 was small, and the study is likely underpowered. However, we believe it important to share all data with the scientific community, especially in light of current debates about transparency and scientific openness [44] 6 . Our measure of physical and emotional satisfaction may also be improved upon.…”

Section: Discussionmentioning

confidence: 99%