Better Understanding the Population Size and Stigmatization of Psychologists Using Questionable Research Practices

Fox, Nicholas William; Honeycutt, Nathan; Jussim, Lee

doi:10.31234/osf.io/3v7hx

Cited by 9 publications

(14 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The lower rates reported in Krishna et al (2018) could also be due to the focus on the undergraduate thesis specifically: in our study, students were reporting on broader research involvement, where there could be more opportunity for QRPs. Further, despite some methodological differences, the rates of self-reported QRP use from this study are broadly comparable to previous surveys of non-student researchers in psychology and related disciplines (Agnoli et al, 2017;Héroux et al, 2017;Janke et al, 2019;John et al, 2012), although some studies report slightly higher (Artino et al, 2019;Chin, 2021;Makel et al, 2021) and lower (Fiedler & Schwarz, 2016;Fox et al, 2018;Rabelo et al, 2020;Wolff et al, 2018) estimates (Table 1). It appears that rates reported by students in the current study are more similar to previous research with non-student researcher samples than to student samples.…”

Section: Discussionsupporting

confidence: 82%

“I know it's bad but I have been pressured into it”: Questionable research practices among psychology students in Canada

Moran¹,

Richard²,

Wilson³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Questionable research practices (QRPs) have been identified as a driving force of the replication crisis in the field of psychological science. The aim of this study was to assess the frequency of QRP use among psychology students in Canadian universities, and to better understand reasons and motivations for QRP use.Method: Participants were psychology students attending Canadian universities and were recruited via online advertising and university email invitations to complete a bilingual survey. Respondents were asked how often they and others engaged in seven QRPs. They were also asked to estimate the proportion of psychology research impacted by each QRP and how acceptable they found each QRP. Data were collected through Likert-scale survey items and open-ended text responses between May 2020 and January 2021, and was analyzed using descriptive statistics and thematic analysis. Results: 425 psychology students completed the survey. The sample consisted of 40% undergraduate students, 59% graduate students and 1% post-doctoral fellows. Overall, 64% of participants reported using at least one QRP, while 79% reported having observed others engaging in at least one QRP. The most frequently reported QRPs were p-hacking (46%), not submitting null results for publication (31%), excluding outcome measures (30%), and hypothesizing after results are known (27%). These QRPs were also the most frequently observed in others, estimated to be the most prevalent in the field, and rated as the most acceptable. Qualitative findings show that students reported that pressures to publish motivated their QRP use, with some reporting that certain QRPs are justifiable in some cases (e.g., in the case of exploratory research). Students also reported that QRPs contribute to the replication crisis and to publication bias and offered several alternatives and solutions to engaging in QRPs, such as gaining familiarity with open science practices. Conclusions: Most Canadian psychology students in this sample report using QRPs, which is unsurprising since they observe such practices in their research environment and estimate that they are prevalent. In contrast, most students believe that QRPs are not acceptable. The results of this study highlight the need to examine the pedagogical standards and cultural norms in academia that may promote or normalize QRPs in psychological science, to improve the quality and replicability of research in this field.

show abstract

Section: Discussionsupporting

confidence: 82%

“I know it's bad but I have been pressured into it”: Questionable research practices among psychology students in Canada

Moran¹,

Richard²,

Wilson³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The exact descriptions of behaviors follow closely, but differ somewhat in Fiedler & Schwarz, 2016;Héroux et al, 2017;Swift et al, 2020;Wolff et al, 2018. The question stems in most studies asked researchers whether they had ever engaged in the focal behavior, fewer studies asked about frequency (Janke et al, 2019;Makel et al, 2019;Motyl et al, 2017), specific projects (Bosco et al, 2016;Krishna & Peter, 2018;Wolff et al, 2018) or over the past year (Fox et al, 2018) S5 QRP behavior Description, with substantive variations in wording across studies noted At least one For studies that asked psychologists about whether they had ever engaged in the ten behaviors from , the proportion that reported engaging in at least one.…”

Section: Retractionsmentioning

confidence: 99%

Replicability, Robustness, and Reproducibility in Psychological Science

Nosek

Hardwicke

Moshontz

et al. 2022

Annu. Rev. Psychol.

403

244

View full text Add to dashboard Cite

Replication—an important, uncommon, and misunderstood practice—is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

show abstract

“…The exact descriptions of behaviors follow (John et al, 2012) closely, but differ somewhat in Fiedler & Schwarz, 2016;Héroux et al, 2017;Swift et al, 2020;Wolff et al, 2018. The question stems in most studies asked researchers whether they had ever engaged in the focal behavior, fewer studies asked about frequency (Janke et al, 2019;Makel et al, 2019;Motyl et al, 2017), specific projects (Bosco et al, 2016;Krishna & Peter, 2018;Wolff et al, 2018) or over the past year (Fox et al, 2018). The reported estimates for Fielder and Schwarz (2016) were extracted from a figure.…”

Section: Retractionsmentioning

confidence: 99%

Replicability, Robustness, and Reproducibility in Psychological Science

Nosek¹,

Hardwicke²,

Moshontz³

et al. 2021

Preprint

View full text Add to dashboard Cite

Replication, an important, uncommon, and misunderstood practice, is making a comeback in psychology. Achieving replicability is a necessary but not sufficient condition for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understanding to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understanding and observed surprising failures to replicate many published findings. Replication efforts also highlighted sociocultural challenges, such as disincentives to conduct replications, framing of replication as personal attack rather than healthy scientific practice, and headwinds for replication contributing to self-correction. Nevertheless, innovation in doing and understanding replication, and its cousins, reproducibility and robustness, have positioned psychology to improve research practices and accelerate progress.

show abstract

Better Understanding the Population Size and Stigmatization of Psychologists Using Questionable Research Practices

Cited by 9 publications

References 42 publications

“I know it's bad but I have been pressured into it”: Questionable research practices among psychology students in Canada

“I know it's bad but I have been pressured into it”: Questionable research practices among psychology students in Canada

Replicability, Robustness, and Reproducibility in Psychological Science

Replicability, Robustness, and Reproducibility in Psychological Science

Contact Info

Product

Resources

About