Editors' perspectives on the selection of reviewers and the quality of reviews

Abstract

This article describes the outcomes of a survey of 93 editors in chief and associate editors of behavior‐analytic journals. We sought information about variables that influence their judgment of the selection of reviewers, selection of review panels, and quality of reviews. When selecting reviewers, participants rated highly expertise on the topic, history of conducting good reviews, and history of writing constructive and respectful reviews. When selecting review panels, participants rated highly stratifying reviewers based on their expertise, avoiding conflicts of interest, and the matching based on the area of expertise between reviewers and authors. When evaluating the quality of a review, participants rated highly considerations related to research design, the science underlying the main idea, and accurate interpretations of the data. Participants did not rate copyediting as important. Overall, the extent to which reviewer selection was influenced by membership in underrepresented groups varied. These findings can inform the development of training programs for teaching peer‐review repertoires.

Reviewing manuscripts for behavior‐analytic journals: A primer

“…Because EICs and AEs invite reviewers to conduct editorial work, judge the quality of reviews, and invite reviewers to serve on editorial boards, training programs for writing reviews should be informed by the expertise and preferences of EICs and AEs. Cengher and LeBlanc (2024) surveyed EICs and AEs of behavioranalytic journals on the variables that influence their judgment of the quality of a review and selection of reviewers. The 93 survey respondents served as EICs, AEs, or guest AEs at least once for a behavior-analytic journal.…”

“…The recommendations for writing reviews for scientific journals included in this article were informed by the results of Cengher and LeBlanc (2024) as well as the editorial experiences of the second author. We grouped these recommendations around four themes: (a) honor your responsibility, (b) know your audience, (c) be constructive and kind, and (d) carefully evaluate the merits of the science (see Table 2 for an outline of the survey items that led to each recommendation).…”

“…Note. These sections can be found in Cengher and LeBlanc (2024), Tables 2 and 3. Unless otherwise indicated in the third column, all questions had multiple response options that could be answered using a 5-point Likert-type scale.…”

“…A third consideration influencing one's decision to accept an invitation to review is perceived conflicts of interest. Indeed, EICs and AEs in behavior analysis commented on the importance of avoiding potential conflicts of interest (Cengher & LeBlanc, 2024). The Ethics Code for Behavior Analysts, published by the Behavior Analyst Certification Board (2020), emphasizes the importance of managing and avoiding conflicts of interest in both clinical work and research.…”

“…We then describe specific guidelines for conducting reviews for scientific journals. These guidelines were informed by the preferences and expertise of EICs and AEs in behavior analysis (Cengher & LeBlanc, 2024).…”