Background
Prior text analysis of R01 critiques suggested that female applicants
may be disadvantaged in NIH peer review, particularly for R01 renewals. NIH
altered its review format in 2009. The authors examined R01 critiques and
scoring in the new format for differences due to principal investigator (PI)
sex.
Method
The authors analyzed 739 critiques—268 from 88 unfunded and
471 from 153 funded applications for grants awarded to 125 PIs (M =
76, 61% F = 49, 39%) at the University of
Wisconsin-Madison between 2010 and 2014. The authors used 7 word categories
for text analysis: ability, achievement, agentic, negative evaluation,
positive evaluation, research, and standout adjectives. The authors used
regression models to compare priority and criteria scores, and results from
text analysis for differences due to PI sex and whether the application was
for a new (Type 1) or renewal (Type 2) R01.
Results
Approach scores predicted priority scores for all PIs’
applications (P<.001); but scores and critiques differed
significantly for male and female PIs’ Type 2 applications.
Reviewers assigned significantly worse priority, approach, and significance
scores to female than male PIs’ Type 2 applications, despite using
standout adjectives (e.g., “outstanding,”
“excellent”) and making references to ability in more of
their critiques (P<.05 for all comparisons).
Conclusions
The authors’ analyses suggest that subtle gender bias may
continue to operate in the post-2009 NIH review format in ways that could
lead reviewers to implicitly hold male and female applicants to different
standards of evaluation, particularly for R01 renewals.