Should Psychology Journals Adopt Specialized Statistical Review?

Hardwicke, Tom E; Frank, Michael C.; Vazire, Simine; Goodman, Steven N.

doi:10.1177/2515245919858428

Cited by 21 publications

(21 citation statements)

References 99 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In time, journals might change their stance, and follow the example of other fields like biomedicine, where the (advantage of) in-depth statistical review is more generally recognized [ 31 ]. Within psychology, we only know of one journal, Meta-Psychology , with such a policy.…”

Section: Discussionmentioning

confidence: 99%

Correction notices in psychology: impactful or inconsequential?

Heyman

Maerten

2020

R. Soc. open sci.

View full text Add to dashboard Cite

Science is self-correcting, or so the adage goes, but to what extent is that indeed the case? Answering this question requires careful consideration of the various approaches to achieve the collective goal of self-correction. One of the most straightforward mechanisms is individual self-correction: researchers rectifying their own mistakes by publishing a correction notice. Although it offers an efficient route to correcting the scientific record, it has received little to no attention from a metascientific point of view. We aim to fill this void by analysing the content of correction notices published from 2010 until 2018 in the three psychology journals featuring the highest number of corrections over that timespan based on the Scopus database (i.e. Psychological Science with N = 58, Frontiers in Psychology with N = 99 and Journal of Affective Disorders with N = 57). More concretely, we examined which aspects of the original papers were affected (e.g. hypotheses, data-analyses, metadata such as author order, affiliations, funding information etc.) as well as the perceived implications for the papers’ main findings. Our exploratory analyses showed that many corrections involved inconsequential errors. Furthermore, authors rarely revised their conclusions, even though several corrections concerned changes to the results. We conclude with a discussion of current policies, and suggest ways to improve upon the present situation by (i) preventing mistakes, and (ii) transparently rectifying those mistakes that do find their way into the literature.

show abstract

Section: Discussionmentioning

confidence: 99%

Correction notices in psychology: impactful or inconsequential?

Heyman

Maerten

2020

R. Soc. open sci.

View full text Add to dashboard Cite

show abstract

“…In biomedical journals, there is no single model for statistical review in peer review strategies [51][52][53]. Some journals recruit statistical methodologists to the editorial board, some draw their statistical reviewers from an external pool.…”

Section: Discussionmentioning

confidence: 99%

Ten Points for High-Quality Statistical Reporting and Data Presentation

Nieminen

2020

Applied Sciences

View full text Add to dashboard Cite

Background: Data analysis methods have become an essential part of empirical research papers, especially in health sciences and medical research. It has previously been reported that a noteworthy percentage of articles have flaws in their statistical reporting. Reporting problems have been a long-term issue, and despite continued efforts to improve the situation, improvements have been far from satisfactory. One explanation is an inadequate assessment of statistical reporting during peer review. This communication proposes a short instrument to assess the quality of data analysis reporting in manuscripts and published papers. Method: A checklist-type instrument was developed by selecting and refining items from previous reports about the quality of statistical reporting in medical journals and from published guidelines for reporting and data presentation. Items were pretested and modified during pilot studies. A total of 160 original medical research articles that were published in 4 journals were evaluated to test the instrument. Interrater and intrarater agreements were examined by comparing quality scores assigned to 40 articles published in a psychiatric journal. Results: The data analysis reporting test consists of nine questions that assess the quality of health research from a reader’s perspective. The composed scale has a total score ranging from 0 to 10 and discriminated between journals and study designs. A high score suggested that an article had a good presentation of findings in tables and figures and that the description of analysis methods was helpful to readers. Interrater and intrarater agreements were high. Conclusion: An applicable checklist for quickly testing the statistical reporting quality of manuscripts and published research papers was developed. This instrument aims to improve the quality of empirical research in scientific fields where statistical methods play an important role.

show abstract

“…Manuscript Evaluation. Editors could also invite specialized reviewers on their editorial board who evaluate and encourage open practices and reproducible methods and statistics (Hardwicke et al, 2019). Similarly, when providing evaluation criteria for reviewers, editors could offer explicit evaluation criteria that emphasise open practices.…”

Section: Journalsmentioning

confidence: 99%

Promoting Open Science: A holistic approach to changing behaviour

Robson¹,

Ma²,

Jl³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this article, we provide a toolbox of resources and nudges for those who are interested in advancing open scientific practice. Open Science encompasses a range of behaviours that aim to include the transparency of scientific research and how widely it is communicated. The paper is divided into seven sections, each dealing with a different stakeholder in the world of research (researchers, students, departments and faculties, universities, academic libraries, journals, and funders). With two frameworks in mind — EAST and the Pyramid of Culture Change — we describe the influences and incentives that sway behaviour for each of these stakeholders, we outline changes that can foster Open Science, and suggest actions and resources for individuals to nudge these changes. In isolation, a small shift in one person’s behaviour may appear to make little difference, but when combined, these small shifts can lead to radical changes in culture. We offer this toolbox to assist individuals and institutions in cultivating a more open research culture.

show abstract

Should Psychology Journals Adopt Specialized Statistical Review?

Cited by 21 publications

References 99 publications

Correction notices in psychology: impactful or inconsequential?

Correction notices in psychology: impactful or inconsequential?

Ten Points for High-Quality Statistical Reporting and Data Presentation

Promoting Open Science: A holistic approach to changing behaviour

Contact Info

Product

Resources

About