Pre-registration: Weighing costs and benefits for researchers

Logg, Jennifer M.; Dorison, Charles

doi:10.1016/j.obhdp.2021.05.006

Cited by 43 publications

(30 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Preregistration of analysis plans allows for a clear timestamped distinction between confirmatory and exploratory hypotheses and analyses (Nosek et al, 2018;Logg & Dorison, 2021).…”

Section: Rationales For Preregistrationmentioning

confidence: 99%

“…It is well known that the human mind is geared towards confirmation bias (seeking to prove ourselves right; Mynatt et al, 1977), a bias to which researchers certainly are not immune (Nelson, 2014). This bias may lead researchers to conduct multiple analyses and subsequently focus only on the findings that support their hypotheses while dismissing/forgetting the ones that did not provide the hoped-for answer (Logg & Dorison, 2021). Analytical decisions made in an exploratory fashion (such as looking for heterogeneous treatment effects across subgroups) have been found to lead to inferences with low validity and a lower likelihood of successful replication of the results (Hainmueller et al, 2019).…”

Section: Rationales For Preregistrationmentioning

confidence: 99%

“…Given the probable inflation of type I errors, if analyses that attempted multiple analyses and parameterizations are presented as confirmatory, readers and researchers may interpret the results of a study as more likely to replicate than what they may deserve (Logg & Dorison, 2021). By preregistering analytical decisions in a timestamped manner, it is made clear what was intended before data analysis had begun, enabling readers to assess what decisions were made after accessing the data.…”

Section: Rationales For Preregistrationmentioning

confidence: 99%

“…By preregistering analytical decisions in a timestamped manner, it is made clear what was intended before data analysis had begun, enabling readers to assess what decisions were made after accessing the data. And, whereas exploratory analyses are welcome and encouraged in preregistered studies, preregistration allows for open and transparent time-logging of which hypotheses and analyses were confirmatory and which were exploratory (Logg & Dorison, 2021). Furthermore, preregistration of analysis plans limits the number of confirmatory tests and keeps the type I error rate at p<0.05.…”

Section: Rationales For Preregistrationmentioning

confidence: 99%

“…As stated before, preregistration does not prevent researchers from conducting exploratory analyses or from learning from the data. What it allows for is a transparent and timestamped approach to delineating confirmatory versus exploratory analyses (Logg & Dorison, 2021). Journals adopting peer review and conditional acceptance of papers before data is collected or analyzed (of which preregistration is a mandatory component) offset any worry that preregistration would limit a researcher's ability to publish.…”

Section: Adopting Preregistration and Replicationmentioning

confidence: 99%

See 4 more Smart Citations

The need for public opinion and survey methodology research to embrace preregistration and replication

Lundmark¹,

Protzko²,

Weissenbilder³

2022

Preprint

View full text Add to dashboard Cite

Survey researchers take great care to measure and report respondents’ answers in an unbiased way; but, how well are we as a field at remedying our own unintended and intended biases in our research? The validity of inferences drawn from studies has been found to be improved by the implementation of preregistration practices. Despite this, only three of the 83 published articles in POQ and IJPOR in 2020 featured explicitly stated preregistered hypotheses or analyses. This manuscript aims to show survey methodologists how preregistration and replication (where possible) are in service to the broader mission of survey methodology. To that end, we also present a practical example of how unknown biases in analysis strategies without preregistration or replication inflate type I errors. In an initial data collection, our analysis shows a certain visual layout of battery-type questions significantly decreases data quality. But after committing to replicating and preregistering the hypotheses and analysis plans, none of the results were successfully replicated, despite keeping the procedure, sample provider, and analyses identical. This manuscript illustrates how preregistration and replication practices will, in the long-term, unburden the academic literature from follow-up publications relying on type I errors.

show abstract

“…Preregistration of analysis plans allows for a clear timestamped distinction between confirmatory and exploratory hypotheses and analyses (Nosek et al, 2018;Logg & Dorison, 2021).…”

Section: Rationales For Preregistrationmentioning

confidence: 99%