Researcher Degrees of Freedom in the Psychology of Religion

Abstract

There is a push in psychology toward more transparent practices, stemming partially as a response to the replication crisis. We argue that the psychology of religion should help lead the way toward these new, more transparent practices to ensure a robust and dynamic subfield. One of the major issues that proponents of Open Science practices hope to address is researcher degrees of freedom (RDF). We pre-registered and conducted a systematic review of the 2017 issues from three psychology of religion journals. We aimed to identify the extent to which the psychology of religion has embraced Open Science practices and the role of RDF within the subfield. We found that many of the methodologies that help to increase transparency, such as pre-registration, have yet to be adopted by those in the subfield. In light of these findings, we present recommendations for addressing the issue of transparency in the psychology of religion and outline how to move toward these new Open Science practices.

Multidimensional Signals and Analytic Flexibility: Estimating Degrees of Freedom in Human-Speech Analyses

“…Combined with a wide variety of methodological and theoretical traditions as well as varying levels of quantitative training across subfields, the inherent flexibility of data analysis might lead to a vast plurality of analytic approaches that can lead to different scientific conclusions (Roettger et al, 2019). Analytic flexibility has been widely discussed from a conceptual point of view (Nosek & Lakens, 2014; Simmons et al, 2011; Wagenmakers et al, 2012) and in regard to its application in individual scientific fields (e.g., Charles et al, 2019; Roettger, 2019; Wicherts et al, 2016). This notwithstanding, there are still many unknowns regarding the extent of analytic plurality in practice.…”

Advancing the Cognitive Science of Religion through Replication and Open Science

“…We refer the interested reader to Crüwell et al (2019) for an accessible introduction and annotated reading list on open science practices in general, and to Charles et al (2019) for an assessment of and recommendations for open science within the psychology of religion in particular (see also Table 1).…”

“…Various labs now preregister their studies (e.g., Gervais et al, 2017;McPhetres, 2018;White et al, 2018), and data sharing has become far more common (e.g., Jong et al, 2019;Purzycki et al, 2018). At the same time, a recent systematic analysis of the 2017 issues from three psychology of religion journals found that none of the 53 included articles were preregistered (Charles et al, 2019).…”

Advancing the Cognitive Science of Religion through Replication and Open Science

“…Various labs now preregister their studies (e.g., Gervais et al, 2017;McPhetres, 2018;White et al, 2018), and data sharing has become far more common (e.g., Jong et al, 2019;Purzycki et al, 2018). At the same time, a recent systematic analysis of the 2017 issues from three psychology of religion journals found that none of the 53 included articles were preregistered (Charles et al, 2019). Furthermore, with regard to replication research, most studies conducted so far were aimed at replicating social psychological or priming studies (e.g., Gomes & McCullough, 2015;Sanchez et al, 2017).…”