Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability.

Marsh, Herbert W.; Jayasinghe, Upali W; Bond, Nigel W.

doi:10.1037/0003-066x.63.3.160

Cited by 252 publications

(235 citation statements)

References 33 publications

(78 reference statements)

Supporting

Mentioning

221

Contrasting

Unclassified

Order By: Relevance

“…Many manuscripts are rejected on this criterion, even if the reviewers identify the research as sound and reported effectively. Despite evidence of the unreliability of the review process for evaluation and identifying importance (Bornmann, Mutz, & Daniel, 2010;Cicchetti, 1991;Gottfredson, 1978;Marsh & Ball, 1989;Marsh, Jayasinghe, & Bond, 2008;Peters & Ceci, 1982;Petty, Fleming, & Fabrigar, 1999;Whitehurst, 1984), this is a reasonable criterion given that journals have limited space and desires to be prestigious outlets. However, in the digital age, page limits are an anachronism (Nosek & Bar-Anan, 2012).…”

Section: Journals With Peer Review Standards Focused On the Soundnessmentioning

confidence: 99%

Scientific Utopia

Nosek

Spies

Motyl

2012

Perspect Psychol Sci

1,075

421

View full text Add to dashboard Cite

An academic scientist's professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive-getting it right-competitive with the more tangible and concrete incentive-getting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.

show abstract

Section: Journals With Peer Review Standards Focused On the Soundnessmentioning

confidence: 99%

Scientific Utopia

Nosek

Spies

Motyl

2012

Perspect Psychol Sci

1,075

421

View full text Add to dashboard Cite

show abstract

“…** We were able to rule out that these evaluations were due to unequal gender representation in the review committee. On average, committees contained 32.6% female reviewers; the gender composition of the review committee was unrelated to application evaluations and success rates for male and female applicants (26,27) (SI Text and Fig. S1).…”

Section: Application Evaluations and Success Ratesmentioning

confidence: 99%

Gender contributes to personal research funding success in The Netherlands

Lee

Ellemers

2015

Proc. Natl. Acad. Sci. U.S.A.

234

166

View full text Add to dashboard Cite

We examined the application and review materials of three calls (n = 2,823) of a prestigious grant for personal research funding in a national full population of early career scientists awarded by the Netherlands Organization for Scientific Research (NWO). Results showed evidence of gender bias in application evaluations and success rates, as well as in language use in instructions and evaluation sheets. Male applicants received significantly more competitive "quality of researcher" evaluations (but not "quality of proposal" evaluations) and had significantly higher application success rates than female applicants. Gender disparities were most prevalent in scientific disciplines with the highest number of applications and with equal gender distribution among the applicants (i.e., life sciences and social sciences). Moreover, content analyses of the instructional and evaluation materials revealed the use of gendered language favoring male applicants. Overall, our data reveal a 4% "loss" of women during the grant review procedure, and illustrate the perpetuation of the funding gap, which contributes to the underrepresentation of women in academia.gender bias | research funding | success rates | academia | STEM W omen are still underrepresented in academia today. Despite various attempts to promote gender equality (e.g., affirmative action initiatives, quotas), female scientists are less likely to get offered tenure, are judged to be less competent, receive less payment and research facilities, and are less likely to be awarded research grants compared with male scientists (1-3). Over time, this type of bias accumulates and contributes to the attrition of women from academia (4); the academic pipeline leaks. Here we report evidence of gender bias in personal research funding for early career scientists.The importance of equal gender representation is widely acknowledged, for several reasons. First, it can resolve historic inequalities and provide equal opportunities (the "moral case for diversity") (5, 6). Second, research has demonstrated that gender diversity in organizations can boost innovation and creativity, expand target groups, and increase productivity (the "business case for diversity") (7, 8), either directly or indirectly through decision making processes or reputation, and provided that the organizational culture is open to change (9-12). This is true for academia as well. Gender-diverse research teams facilitate innovation and excellence in research and policy, address different research questions and methods, and facilitate wider application of research findings (13), thereby contributing to scientific progress.Relevant statistics, however, show a persistent leadership gap, salary gap, and funding gap for women in academia (3). To illustrate, The Netherlands-generally considered a high-equity nationhad only 16% female full professors in 2012 (14). Leadership is stereotyped as masculine and consequently associated more strongly with stereotypical male traits rather than female traits. Although women a...

show abstract

“…It means that published journal articles are valuable in a very concrete manner. They also serve as merits when applying for research funding or starting research projects (Marsh, Jayasinghe, & Bond, 2008). The term "publish or perish" has become more than familiar to researchers across the world (e.g., Ching, 2013;Roth 2002).…”

Section: The Pressure Of Publishingmentioning

confidence: 99%

Scientific publishing as the arena of power and caring

Uusiautti¹

2015

IJRSE

View full text Add to dashboard Cite

Scientific publishing has become one of the most important means of not only advancing researchers' careers but also securing funding for universities. Therefore, the atmosphere in academic communities often appears very competitive, which can harm trust and open collaboration between colleagues. What is the role of co-authorship in writing? How to perceive the peer review process as a part of publishing and scientific discourse? In this article, scientific publishing of research studies in peer reviewed journals is viewed as an arena of power and caring. This viewpoint has basically two purposes: (1) to discuss the possibility of caring review process and (2) to discuss the possibility of caring collaboration between researchers in scientific writing. The power of collaboration and benefits of caring reviews are introduced as conclusions.

show abstract

Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability.

Cited by 252 publications

References 33 publications

Scientific Utopia

Scientific Utopia

Gender contributes to personal research funding success in The Netherlands

Scientific publishing as the arena of power and caring

Contact Info

Product

Resources

About