Toward Transparent Reporting of Psychological Science

The replication crisis: How might philosophy and theory of psychology be of use?

“…And not just talk. The calamity has motivated large-scale programs to rectify the perceived problems; these include reforms of preexperimental practices, replication efforts, journal editorial policies, and open access to data and publications (see LeBel & John, 2017; Nosek & Bar-Anan, 2012; Nosek, Ebersole, DeHaven, & Mellor, 2018; Nosek, Spies, & Motyl, 2012; Open Science Collaboration, 2012). It has captivated the attention of granting agencies, professional organizations, journals, and academic institutions, many of which are introducing new regulatory measures or revising existing ones.…”

Psychologists’ psychologies of psychologists in a time of crisis.

“…As Heene and Ferguson (2017) report, scientific practices are not intentionally dishonest but rather involve humans doing what they naturally do, namely, engaging in confirmation bias—i.e., valuing evidence that supports their beliefs over evidence that does not, and nudging the statistical system until they find the results they want or expect to see. (p. 45) Confirmation bias is one of the “psychological obstacles” to transparent reporting, according to Etienne LeBel and Leslie John (LeBel & John, 2017); it tops a list that includes “motivated reasoning” and “goal gradients,” all of which are associated with temptations to cheat. Some authors merely name this bias (Carlsson et al, 2017; Ioannidis, 2012; Strack, 2017), remark that it can be conscious and unconscious (Bohannon, 2015), or that it occurs in editors as well as researchers (Bohannon, 2015; Hales, 2016; Nelson et al, 2018).…”

“…A competitive scientific culture that rewards high productivity and novel findings significantly influences psychologists’ motivation, and this culture has created and sustains incentives for undertaking certain kinds of projects and maximizing certain outcomes (Fuchs, Jenny, & Fiedler, 2012; Jussim et al, 2016; Koole & Lakens, 2012; Levenson, 2017; Maner, 2014; Nosek et al, 2018; Nosek, Spies, & Motyl, 2012). Some see the incentive problem from another angle: Instead of identifying the wrong incentives present in contemporary science, they assert that there are insufficient positive incentives, especially in regard to undertaking replications (Earp & Trafimow, 2015; Frank & Saxe, 2012; LeBel & John, 2017). These accounts adopt economic language, analyzing the situation with terms like incentives , investment of resources , opportunities , costs , self-interest , payoff , maximizing success , capitalizing , and rewards .…”

“…The stunning fraud of Diederik Stapel along with several less-publicized incidents disturbed the scientific community, prompting a cascade of autopsies, commentaries, and recommendations for detecting scientific dishonesty (Derksen, 2019). Fraud understandably enters some of the conversations about replication: Some connect fraudulent practices with inadequate vigilance in evaluating research and with the slippery slope of questionable research practices (Fanelli, 2009; LeBel & John, 2017; Neuroskeptic, 2012; Wicherts, 2011). However, a number of commentators take a broader perspective on scientific dishonesty.…”

“…As Simonsohn (2013) observed, science’s “utopian premise” of honesty is inconsistent with reality (p. 1876). The problem of honesty lies with a scientific culture of secrecy, nontransparency, and inadequate scrutiny (LeBel & John, 2017; Simonsohn, 2013; Wicherts, 2011). Even more skeptical is Stroebe, Postmes, and Spears’s (2012) observation that supposed prophylactics against fraud, such as replication, meta-analysis, and peer review, do not work, citing evidence that, in psychology, most frauds have been detected by whistleblowers.…”