Exploring Bias in Scientific Peer Review: An ASCO Initiative

Abstract

PURPOSE: To investigate implicit bias (IB) in the peer review process across ASCO and Conquer Cancer Foundation and to propose potential mitigation strategies. MATERIALS AND METHODS: We, ASCO Working Group on Implicit Bias, selected four data sources: (1) literature search [(a) defining IB in peer review, (b) evidence of IB in peer review, and (c) strategies to mitigate IB]; (2) created and analyzed an ASCO database for sex, race, and institutional affiliation regarding peer review success; (3) constructed and conducted qualitative interviews of key stakeholders within the ASCO board, publications, and grants committee, on experience with IB within ASCO; and (4) constructed, delivered, and analyzed results of member survey on perception of IB within ASCO. RESULTS: Historically uncommon, PubMed articles on IB in peer review subsequently increased exponentially in the past 2 decades. Qualitative interviews of ASCO key stakeholders reveal that system changes and IB training were priorities. The committee member survey reported that their peer review decisions could be affected by IB and that mitigating IB should be a priority. Most reported having never been trained on IB. Available data from ASCO database support stakeholder findings, suggesting that there exists a disproportionate representation of males and better-known institutions among both reviewer positions and awardees. Ethnicity/race data were insufficiently reported. Limited data on interventions/strategies to mitigate IB in the peer-reviewed literature suggest that there are feasible processes for grants, program committees, and journals. CONCLUSION: Limited data reveal that the peer review process at ASCO is not exempt from IB and suggest association with sex and institutional affiliation. Working Group on Implicit Bias recommends three actions to mitigate IB within peer review: (1) create awareness and a culture of inclusivity, (2) create systems to reduce IB, and (3) collect data for ongoing analysis.

Influence of participant and reviewer characteristics in application scores for a hematology research training program

“…11 Our ability to analyze race and ethnicity by applicant and reviewer is important, as previous efforts to evaluate these characteristics were limited because of lack of availability of these data elements. 10,12 We did not find that application scores for female applicants differed from male applicants. In contrast to our finding, bias by gender in academic settings has been observed during grant peer review.…”

“… 11 Our ability to analyze race and ethnicity based on the applicant and reviewer is important because previous efforts to evaluate these characteristics were limited owing to the lack of availability of these data elements. 10 , 12 …”

“…In contrast to our finding, bias based on sex in academic settings has been observed during grant peer review. 12 , 13 In addition, manuscripts with female first authors received significantly lower scores in peer review and were cited less often compared with those with male first authors. 14 Although we found that women and men had similar application scores, our other research has demonstrated that female CRTI alumnae experienced less academic success, as measured by publications and protected time for research.…”

Gender and geographic representation in editorial boards of education journals

“…Geographical diversity encompasses a multitude of viewpoints stemming from distinct geographic regions, cultural contexts, and socio-economic environments. The absence of diverse geographical representation within an editorial committee can inadvertently sustain bias and ethnocentrism in the selection and evaluation of scholarly work ( McKenzie et al, 2022 ). To address this issue, internationalizing editorial boards can help mitigate potential implicit biases.…”

How to be an effective ally

“…When reviewing manuscripts, pay attention to the other reviewer comments and make sure that bias is not occurring or if you see it, let the editor know. Implicit bias occurs most frequently when sex or institutional affiliation are known to the reviewers (McKenzie, et al, 2022; Verharen, 2023) and there is evidence that double blinding the review process can mitigate implicit gender bias (Kern‐Goldberger et al, 2022). Educate others in your field and workplace about issues of diversity, equity, and inclusion and why these concepts are important.…”