Justify your alpha

Lakens, Daniël; Adolfi, Federico; Albers, Casper J.; Anvari, Farid; Apps, Matthew A. J.; Argamon, Shlomo; Baguley, Thom; Becker, Raymond; Benning, Stephen D.; Bradford, Daniel E.; Buchanan, Erin Michelle; Caldwell, Aaron R.; Calster, Ben Van; Carlsson, Rickard; Chen, Sau-Chin; Chung, Bryan; Colling, Lincoln; Collins, Gary S.; Crook, Zander; Cross, Emily S.; Daniels, Sameera; Danielsson, Henrik; DeBruine, Lisa; Dunleavy, Daniel J.; Earp, Brian D.; Feist, Michele I.; Ferrell, Jason D.; Field, James G.; Fox, Nicholas W.; Friesen, Amanda; Gomes, Caio; González-Márquez, Mónica; Grange, James A.; Grieve, Andrew P.; Guggenberger, Robert; Grist, James T.; Harmelen, Anne‐Laura van; Hasselman, Fred; Hochard, Kevin D.; Hoffarth, Mark R.; Holmes, Nicholas P.; Ingre, Michael; Isager, Peder Mortvedt; Isotalus, Hanna K.; Johansson, Christer; Juszczyk, Konrad; Kenny, David A.; Khalil, Ahmed A.; Konat, Barbara; Lao, Junpeng; Larsen, Erik Gahner; Lodder, Gerine M. A.; Lukavský, Jiřı́; Madan, Christopher R.; Manheim, David; Martin, Stephen R.; Martin, Andrea E.; Mayo, Deborah G.; McCarthy, Randy J.; McConway, Kevin; McFarland, Colin; Nio, Amanda Q. X.; Nilsonne, Gustav; Oliveira, Cilene Lino de; Xivry, Jean-Jacques Orban de; Parsons, Sam; Pfuhl, Gerit; Quinn, Kimberly A.; Sakon, John J.; Sarıbay, S. Adil; Schneider, Iris K.; Selvaraju, Manojkumar; Sjoerds, Zsuzsika; Smith, Samuel G.; Smits, Tim; Spies, Jeffrey R.; Sreekumar, Vishnu; Steltenpohl, Crystal N.; Stenhouse, Neil; Świątkowski, Wojciech; Vadillo, Miguel A.; Assen, Marcel A.L.M. van; Williams, Matt; Williams, Samantha E.; Williams, Donald R.; Yarkoni, Tal; Ziano, Ignazio; Zwaan, Rolf A.

doi:10.1038/s41562-018-0311-x

Cited by 371 publications

(186 citation statements)

References 12 publications

Supporting

Mentioning

168

Contrasting

Order By: Relevance

“…Especially in clinical research, future trials would need to be larger, less feasible, and more expensive. Achieving 80% power with a threshold of 0.005, instead of 0.05, would require a 70% larger sample size for betweensubject study designs with two-sided tests (88% for onesided tests) [17]. Researchers could abandon some good ideas with the net effect of depressing spontaneous, investigator-initiated research.…”

Section: The Threshold For Significance and Its Originmentioning

confidence: 99%

“…An article signed by 54 authors [18] has provided a similar view, with a deeper technical explanation on why the statements by Ioannidis [15] and Benjamin et al [4] are unjustified. Another article, signed by 88 authors [17], has questioned the idea that the significance threshold should be based on the amount of relative evidence indicated by Bayesian factors 1 , as done by Benjamin et al [4], whose assumptions were considered to be unjustified. A major point deserving further comments is the lack of replicability/reproducibility of study results [4].…”

Section: The Threshold For Significance and Its Originmentioning

confidence: 99%

See 1 more Smart Citation

Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach

2020

View full text Add to dashboard Cite

Here, we summarise the unresolved debate about p value and its dichotomisation. We present the statement of the American Statistical Association against the misuse of statistical significance as well as the proposals to abandon the use of p value and to reduce the significance threshold from 0.05 to 0.005. We highlight reasons for a conservative approach, as clinical research needs dichotomic answers to guide decision-making, in particular in the case of diagnostic imaging and interventional radiology. With a reduced p value threshold, the cost of research could increase while spontaneous research could be reduced. Secondary evidence from systematic reviews/metaanalyses, data sharing, and cost-effective analyses are better ways to mitigate the false discovery rate and lack of reproducibility associated with the use of the 0.05 threshold. Importantly, when reporting p values, authors should always provide the actual value, not only statements of "p < 0.05" or "p ≥ 0.05", because p values give a measure of the degree of data compatibility with the null hypothesis. Notably, radiomics and big data, fuelled by the application of artificial intelligence, involve hundreds/thousands of tested features similarly to other "omics" such as genomics, where a reduction in the significance threshold, based on well-known corrections for multiple testing, has been already adopted.

show abstract

Section: The Threshold For Significance and Its Originmentioning

confidence: 99%

Section: The Threshold For Significance and Its Originmentioning

confidence: 99%

Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach

2020

View full text Add to dashboard Cite

show abstract

“…Although the authors acknowledge the problems of NHST, they argue that such a change would solve many of the problems that lead to low replicability (e.g., by making p ‐hacking and QRPs harder to work) and would be easy to implement. In response, other 88 authors argue for a more critical approach in which authors should be required to specify and justify the significance level that their project needs (Lakens et al, ).…”

Section: What To Do?mentioning

confidence: 99%

Philosophy of science and the replicability crisis

Romero

2019

Philosophy Compass

View full text Add to dashboard Cite

Replicability is widely taken to ground the epistemic authority of science. However, in recent years, important published findings in the social, behavioral, and biomedical sciences have failed to replicate, suggesting that these fields are facing a “replicability crisis.” For philosophers, the crisis should not be taken as bad news but as an opportunity to do work on several fronts, including conceptual analysis, history and philosophy of science, research ethics, and social epistemology. This article introduces philosophers to these discussions. First, I discuss precedents and evidence for the crisis. Second, I discuss methodological, statistical, and social‐structural factors that have contributed to the crisis. Third, I focus on the philosophical issues raised by the crisis. Finally, I discuss several proposals for solutions and highlight the gaps that philosophers could focus on.

show abstract

“…McKiernan and colleagues (2016) also highlight new funding opportunities for open research and discuss the possibility of improved career outcomes through increased opportunities for finding collaborators. Indeed, the first author has found two largescale collaborative opportunities through Twitter alone (Lakens et al, 2018;Moshontz et al, 2018). With open access, researchers can more easily find one another and collaborate, increasing the chances of the data we collect being used.…”

Section: Current Context For Community Psychologists In Open Access Pmentioning

confidence: 99%

Giving Community Psychology Away: A case for open access publishing

Cn¹,

Aj²,

Daniels³

2020

Preprint

View full text Add to dashboard Cite

Author Biographies: Crystal N. Steltenpohl, is an assistant professor of psychology and founder of the Online Technologies Lab at the University of Southern Indiana. As a scholar, gamer, and participant in online worlds, her major research interests revolve around how we interact with various technologies, especially those that house online communities. She is particularly interested in understanding how technologies influence communities, the self, and health behaviors. Amy J. Anderson, is a doctoral candidate in community psychology at DePaul University. Her research focuses on positive youth development and educational equity. Specifically, she is interested in the roles of identity and supportive adult relationships on youth educational experiences. In addition, she is interested in public policy solutions to increase educational access. Katherine M. Daniels, is an assistant professor of psychology at the University of Southern Indiana. Her major research interests include animal cognition and student learning. She and Dr. Steltenpohl head an effort to create an open source video game that teaches high school and undergraduate students about research methods. AbstractAmidst increased pressure for transparency in science, researchers and community members are calling for open access to study stimuli and measures, data, and results. These arguments coincidentally align with calls within community psychology to find innovative ways to support communities and increase the prominence of our field. This paper aims to (1) define the current context for community psychologists in open access publishing, (2) illustrate the alignment between open access publishing and community psychology principles, and (3) demonstrate how to engage in open access publishing using community psychology values. Currently, there are several facilitators (e.g. an increasing number of open access journals, the proliferation of blogs, and social media) and barriers (e.g. Article Processing Charges (APCs), predatory journals) to publishing in open access venues. Openly sharing our research findings aligns with our values of (1) citizen participation, (2) social justice, and (3) collaboration and community strengths. Community psychologists desiring to engage in open access publishing can ask journals to waive APCs, publish pre-prints, use blogs and social media to share results, and push for systemic change in a publishing system that disenfranchises researchers, students, and community members."I can imagine nothing we could do that would be more relevant to human welfare [...] than to discover how best to give psychology away." -George A. Miller (1969)

show abstract

Justify your alpha

Cited by 371 publications

References 12 publications

Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach

Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach

Philosophy of science and the replicability crisis

Giving Community Psychology Away: A case for open access publishing

Contact Info

Product

Resources

About