Reviewer bias in single- versus double-blind peer review

Tomkins, Andrew; Zhang, Min; Heavlin, William D.

doi:10.1073/pnas.1707323114

Cited by 421 publications

(406 citation statements)

References 21 publications

Supporting

Mentioning

395

Contrasting

Unclassified

Order By: Relevance

“…Our finding that the gender of the last authors was associated with a significant difference in 614 the rate at which full submissions were accepted at eLife stands in contrast with a number of 615 previous studies of journal peer review that reported no significant difference in outcomes of 616 papers submitted by male and female authors [85][86][87], or differences in reviewer's evaluations 617 based on the author's apparent gender [88]. This discrepancy may be explained in part by 618 eLife's unique context, policies, or the relative selectivity of eLife compared to journals where 619 previous studies found gender equity.…”

contrasting

confidence: 84%

Author-Reviewer Homophily in Peer Review

Murray

Siler

Larivière

et al. 2018

Preprint

137

151

View full text Add to dashboard Cite

The fairness of scholarly peer review has been challenged by evidence of disparities in publication outcomes based on author demographic characteristics. To assess this, we conducted an exploratory analysis of peer review outcomes of 23,876 initial submissions and 7,192 full submissions that were submitted to the biosciences journal eLife between 2012 and 2017. Women and authors from nations outside of North America and Europe were underrepresented both as gatekeepers (editors and peer reviewers) and authors. We found evidence of a homophilic relationship between the demographics of the gatekeepers and authors and the outcome of peer review; that is, there were higher rates of acceptance in the case of gender and country homophily. The acceptance rate for manuscripts with male last authors was seven percent, or 3.5 percentage points, greater than for female last authors (95% CI = [0.5, 6.4]); this gender inequity was greatest, at nine percent or about 4.8 percentage points (95% CI = [0.3, 9.1]), when the team of reviewers was all male; this difference was smaller and not significantly different for mixed-gender reviewer teams. Homogeny between countries of the gatekeeper and the corresponding author was also associated with higher acceptance rates for many countries. To test for the persistence of these effects after controlling for potentially confounding variables, we conducted a logistic regression including document and author metadata. Disparities in acceptance rates associated with gender and country of affiliation and the homophilic associations remained. We conclude with a discussion of mechanisms that could contribute to this effect, directions for future research, and policy implications. Code and anonymized data have been made available at https://github.com/murrayds/elife-analysis 1/61 Submitted to PLOS Biology Author summaryPeer review, the primary method by which scientific work is evaluated, is ideally a fair and equitable process in which scientific work is judged solely on its own merit. However, the integrity of peer review has been called into question based on evidence that outcomes often differ between male and female authors, and for authors in different countries. We investigated such disparities at the biosciences journal eLife by analyzing the demographics of authors and gatekeepers (editors and peer reviewers), and peer review outcomes of all submissions between 2012 and 2017. Outcomes were more favorable for male authors and those affiliated with institutions in North America and Europe; these groups were also over-represented among gatekeepers. There was evidence that peer review outcomes were influenced by homophily-a preference of gatekeepers for manuscripts from authors with shared characteristics. We discuss mechanisms that could contribute to this effect, directions for future research, and policy implications. 2/61Submitted to PLOS Biology 1 Peer review is foundational to the development, gatekeeping, and dissemination of research, 2 while also underpinning the professional hiera...

show abstract

contrasting

confidence: 84%

Author-Reviewer Homophily in Peer Review

Murray

Siler

Larivière

et al. 2018

Preprint

137

151

View full text Add to dashboard Cite

show abstract

“…However, peer review may also be subject to systemic biases that influence editorial outcomes (Lee, Sugimoto, Zhang, & Cronin, ). For example, reviewers rate papers with famous authors, or authors from prestigious institutions, more highly (Tomkins, Zhang, & Heavlin, ). Editors and reviewers may also exhibit biases, conscious, or unconscious, against authors who speak a different language or reside in a different country from themselves (Lee et al, ; Murray et al, ).…”

Section: Introductionmentioning

confidence: 99%

Gender differences in peer review outcomes and manuscript impact at six journals of ecology and evolution

Fox

Paine

2019

Ecology and Evolution

147

104

View full text Add to dashboard Cite

The productivity and performance of men is generally rated more highly than that of women in controlled experiments, suggesting conscious or unconscious gender biases in assessment. The degree to which editors and reviewers of scholarly journals exhibit gender biases that influence outcomes of the peer‐review process remains uncertain due to substantial variation among studies. We test whether gender predicts the outcomes of editorial and peer review for >23,000 research manuscripts submitted to six journals in ecology and evolution from 2010 to 2015. Papers with female and male first authors were equally likely to be sent for peer review. However, papers with female first authors obtained, on average, slightly worse peer‐review scores and were more likely to be rejected after peer review, though the difference varied among journals. These gender differences appear to be partly due to differences in authorial roles. Papers for the which the first author deferred corresponding authorship to a coauthor (which women do more often than men) obtained significantly worse peer‐review scores and were less likely to get positive editorial decisions. Gender differences in corresponding authorship explained some of the gender differences in peer‐review scores and positive editorial decisions. In contrast to these observations on submitted manuscripts, gender differences in peer‐review outcomes were observed in a survey of >12,000 published manuscripts; women reported similar rates of rejection (from a prior journal) before eventual publication. After publication, papers with female authors were cited less often than those with male authors, though the differences are very small (~2%). Our data do not allow us to test hypotheses about mechanisms underlying the gender discrepancies we observed, but strongly support the conclusion that papers authored by women have lower acceptance rates and are less well cited than are papers authored by men in ecology.

show abstract

“…(Tamblyn, Girard, Qian, & Hanley, 2018), that we are subject to conscious and unconscious biases (Fay, 1993;Steinpreis, Anders, & Ritzke, 1999). The evidence on bias in peer review is mixed, with some studies concluding that bias occurs (Link, 1998;Tomkins, Zhang, & Heavlin, 2017;Tushingham, Fulkerson, & Hill, 2017), and other studies finding no significant impact (Lee, 2012), for example, Burns and Fox (2017) concluded that socioeconomics and language are more important factors than editor or reviewer bias in explaining geographic variation in acceptance rates ; and see also Engqvist & Frommen, 2008). Authorsuggested reviewers have been found to be more favourable than editor-selected reviewers (Bornmann & Daniel, 2010;Fox, Burns, Muncy, Meyer, & Thompson, 2017).…”

Section: F4 -Confidentialitymentioning

confidence: 99%

What does better peer review look like? Underlying principles and recommendations for better practice

et al. 2019

View full text Add to dashboard Cite

We conducted a literature review of best practice in peer review. Following this research, we identified five principles for better peer review: Content Integrity, Content Ethics, Fairness, Usefulness, and Timeliness. For each of these principles, we have developed a set of recommendations to improve peer review standards. In this article, we describe the role of peer review and how our five principles support that goal. This article is intended to continue the conversation about improving peer review standards and provide guidance to journal teams looking to improve their standards. It is accompanied by a detailed checklist, which could be used by journal teams to assess their current peer review standards.

show abstract

Reviewer bias in single- versus double-blind peer review

Cited by 421 publications

References 21 publications

Author-Reviewer Homophily in Peer Review

Author-Reviewer Homophily in Peer Review

Gender differences in peer review outcomes and manuscript impact at six journals of ecology and evolution

What does better peer review look like? Underlying principles and recommendations for better practice

Contact Info

Product

Resources

About