Reproducibility blues

Pulverer, Bernd

doi:10.15252/embj.201570090

“…Problems such as phacking, "HARKing" and the "file-drawer" effect have been discussed in science for many years with mixed opinions and widespread debate (e.g. Bissell 2013;Bohannon 2014;Kahneman 2014;Schnall 2014;Fischer 2015;Pulverer 2015). Recent proposals such as guidelines against the mis-use of p-values (Wasserstein & Lazar 2016), banning the p-value (Trafimow & Marks 2015), statistical checking software (Epskamp & Nuijten, 2016), redefining statistical significance (Benjamin et al, 2018), justifying your alpha (Lakens et al, 2018), pre-registering methods (Chambers et al 2014;van 't Veer & Giner-Sorolla, 2016) and the Open Science movement generally (e.g.…”

mentioning

confidence: 99%

Cumulative Science via Bayesian Posterior Passing

Brand

¹

,

Ounsley

²

,

Post

³

et al. 2019

View full text Add to dashboard Cite

This paper introduces a statistical technique known as “posterior passing” in which the results of past studies can be used to inform the analyses carried out by subsequent studies. We first describe the technique in detail and show how it can be implemented by individual researchers on an experiment by experiment basis. We then use a simulation to explore its success in identifying true parameter values compared to current statistical norms (ANOVAs and GLMMs). We find that posterior passing allows the true effect in the population to be found with greater accuracy and consistency than the other analysis types considered. Furthermore, posterior passing performs almost identically to a data analysis in which all data from all simulated studies are combined and analysed as one dataset. On this basis, we suggest that posterior passing is a viable means of implementing cumulative science. Furthermore, because it prevents the accumulation of large bodies of conflicting literature, it alleviates the need for traditional meta-analyses. Instead, posterior passing cumulatively and collaboratively provides clarity in real time as each new study is produced and is thus a strong candidate for a new, cumulative approach to scientific analyses and publishing.

show abstract

“…It is also part of a broader multi‐pronged effort at EMBO Press to improve access, reproducibility, and utility of research data in scientific papers. This includes the recent introduction of Structured Methods (Polychronidou, ), the implementation of data quality and integrity checks (Pulverer, ), and the development of the SourceData project that makes the data behind figures searchable and accessible (http://sourcedata.io, Liechti et al , ). In a subsequent step, accession numbers to source data associated with figures will automatically be included in the Data Availability Section, thanks to the integration of SourceData with the BioStudies database (http://bit.ly/2G0Cn8R).…”

Section: What's Next?mentioning

confidence: 99%

Data citation: what, when, why?

Lemberger

¹

2018

Molecular Systems Biology

2

0

View full text Add to dashboard Cite

show abstract

“…Problems such as p-hacking, "HARKing" and the "filedrawer" effect have been discussed in science for many years with mixed opinions and widespread debate (e.g. Bissell 2013;Bohannon 2014;Kahneman 2014;Schnall 2014;Fischer 2015;Pulverer 2015). Recent proposals such as guidelines against the mis-use of p-values (Wasserstein & Lazar 2016), banning the p-value (Trafimow & Marks 2015), statistical checking software (Epskamp & Nuijten, 2016), redefining statistical significance (Benjamin et al, 2018), justifying your alpha (Lakens et al, 2018), pre-registering methods (Chambers et al 2014;van 't Veer & Giner-Sorolla, 2016) and the Open Science movement generally (e.g.…”

Section: Introductionmentioning

confidence: 99%

Cumulative science via Bayesian posterior passing, an introduction.

Brand¹,

Ounsley²,

Post³

et al. 2017

Preprint

View full text Add to dashboard Cite

This paper introduces a statistical technique known as “posterior passing” in which the results of past studies can be used to inform the analyses carried out by subsequent studies. We first describe the technique in detail and show how it can be implemented by individual researchers on an experiment by experiment basis. We then use a simulation to explore its success in identifying true parameter values compared to current statistical norms (ANOVAs and GLMMs). We find that posterior passing allows the true effect in the population to be found with greater accuracy and consistency than the other analysis types considered. Furthermore, posterior passing performs almost identically to a data analysis in which all data from all simulated studies are combined and analysed as one dataset. On this basis, we suggest that posterior passing is a viable means of implementing cumulative science. Furthermore, because it prevents the accumulation of large bodies of conflicting literature, it alleviates the need for traditional meta-analyses. Instead, posterior passing cumulatively and collaboratively provides clarity in real time as each new study is produced and is thus a strong candidate for a new, cumulative approach to scientific analyses and publishing.

show abstract

Reproducibility blues

Cited by 27 publications

References 13 publications

Cumulative Science via Bayesian Posterior Passing

Cumulative Science via Bayesian Posterior Passing

Data citation: what, when, why?

Cumulative science via Bayesian posterior passing, an introduction.

Contact Info

Product

Resources

About