INTRODUCTIONDuring NIH peer review meetings (Bstudy sections^), scientists discuss and assign Bpriority scores^to grant applications that largely determine funding outcomes. Although the final priority score is an average of each panelist's score, their individual score is anchored to the scores declared publicly by those scientists (usually three) assigned to review and report on the grant application in detail. We have identified Bscore calibration talk^(SCT), a discourse practice where a study section member discusses and interprets the scoring rather than the content of a grant application. We found two forms: self-initiated SCT, when a panelist provides commentary about their own scoring (e.g., BSo I gave it a four, which was probably generous^); and other-initiated SCT, when a panelist challenges the scoring of an assigned reviewer (e.g., BYeah, that was generous.^). Only other-initiated SCT correlated with changes from the initial to the final score among the assigned reviewers of NIH R01 applications. 1 To gain insights into which interactional patterns accompanying SCT influence score change, we examined in detail the five cases of SCT followed by immediate declaration of a score change.
METHODSMeticulously adhering to NIH practices, we constructed and video-recorded four study sections where groups of 8-12 oncology researchers evaluated the same 25 R01 grant applications previously submitted to NIH. Pier et al. provides a full description of our methodology. 1 Participant-reviewers evaluated R01 applications previously reviewed by study sections within NIH's National Cancer Institute. Applications were donated by Principal Investigators identified using NIH's public access database, then de-identified and re-identified by the research team. All applications had been funded either on the first submission or after revision between October 1, 2012 and September 31, 2015.We transcribed all discourse with attention to verbal and non-verbal actions, timing of pauses, marking of overlaps, and details of sound production, including laughter. We identified the speaker who initiated SCT, the assigned reviewer who was the target, and whether or not the target announced a score change. We coded and analyzed data as a case series, 2 treating score change as the Bconditionû nder study and calculating the odds ratio-given Bexposure^to identified patterns of discourse associated with SCT. 2
RESULTSWe identified 15 cases of other-directed SCT, five featuring immediate, public score change (Table 1). Each announced score change was preceded either by shared laughter among study section members (N = 4) or SCT by the chair of the study section (N = 1). There were no cases of immediate score change when a panelist initiated SCT but no shared laughter ensued (7/15). No instances of score change in conjunction with SCT occurred without either laughter or the chair's initiation of SCT.Odds ratios indicated that score change was significantly more likely when laughter was present during SCT than when it was absent (OR = 16; 95%...