Untitled

Clavería, L. E.; Güallar, Eliseo; Camı́, Jordi; Conde, J.; Pastor, Roberto Viciano; Ricoy, José Ramón; Rodrı́guez-Farré, Eduard; Ruiz-Palomo, Fernando; Muñoz, Emilio Muñoz

doi:10.1023/a:1005609624130

Cited by 17 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Retrospective cohort studies are common in the medical sciences (Basile et al 2018;Isogai et al 2017) and have been conducted in the field of Scientometrics (Lokker et al 2011;Cramond et al 2016;Clavería et al 2000). We performed a retrospective cohort study with articles published in New Journal of Physics during 2010 and 2016.…”

Section: Data Collectionmentioning

confidence: 99%

The impact of video abstract on citation counts: evidence from a retrospective cohort study of New Journal of Physics

et al. 2019

View full text Add to dashboard Cite

In this paper, we addressed the question of whether a video abstract of an article affects its citation counts. A retrospective cohort study was conducted using the research articles published in New Journal of Physics during 2010 and 2016. Articles with video abstract (N = 315) as the experimental group, were matched 1:2, with articles without video abstract (N = 630) as the control group, by the same publishing issue, same article type. Specifically, the articles lacking video abstract that appeared immediately before and after each experimental group article were included in the control group. A negative binomial regression model was employed to analyze the data. After controlling for the characteristics of articles (including the number of authors, international co-authorship, character counts of title, character counts of text-based abstract, keyword counts, reference counts, page counts and funding), our results showed that articles with video abstract (experimental group) compared to the articles without video abstract (control group) were expected to have a rate 1.206 times greater for citation counts. This study suggests that a video abstract can potentially serve as a useful genre of a research article for receiving more citation counts.

show abstract

Section: Data Collectionmentioning

confidence: 99%

The impact of video abstract on citation counts: evidence from a retrospective cohort study of New Journal of Physics

et al. 2019

View full text Add to dashboard Cite

show abstract

“…A few studies included in this review have looked at peer review evaluation of post-funding performance and quality (Claveria et al, 2000;Mutz et al, 2015) and compared it to the ex-ante evaluation of proposals; both of these findings observed significant predictive validity of the review decisions (although the work of Mutz relies strongly on some methodological assumptions and may not represent an independent observation). Post-funding evaluations of productivity and impact likely take into account contextual factors of the research that are not represented in bibliometric numbers.…”

Section: Post-funding Review Of Outcomesmentioning

confidence: 99%

External Tests of Peer Review Validity Via Impact Measures

Gallo

Glisson

2018

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

Peer review is used commonly across science as a tool to evaluate the merit and potential impact of research projects and make funding recommendations. However, potential impact is likely to be difficult to assess ex-ante; some attempts have been made to assess the predictive accuracy of these review decisions using impact measures of the results of the completed projects. Although many outputs, and thus potential measures of impact, exist for research projects, the overwhelming majority of evaluation of research output is focused on bibliometrics. We review the multiple types of potential impact measures with an interest in their application to validate review decisions. A review of the current literature on validating peer review decisions with research output impact measures is presented here; only 48 studies were identified, about half of which were US based and sample size per study varied greatly. 69% of the studies employed bibliometrics as a research output. While 52% of the studies employed alternative measures (like patents and technology licensing, post-project peer review, international collaboration, future funding success, securing tenure track positions, and career satisfaction), only 25% of all projects used more than one measure of research output. Overall, 91% of studies with unfunded controls and 71% of studies without such controls provided evidence for at least some level of predictive validity of review decisions. However, several studies reported observing sizable type I and II errors as well. Moreover, many of the observed effects were small and several studies suggest a coarse power to discriminate poor proposals from better ones, but not amongst the top tier proposals or applicants (although discriminatory ability depended on the impact metric). This is of particular concern in an era of low funding success, where many top tier proposals are unfunded. More research is needed, particularly in integrating multiple types of impact indicators in these validity tests, as well as considering the context of the research outputs relative to goals of the research program and concerns for reproducibility, translatability and publication bias. In parallel, more research is needed focusing on the internal validity of review decision making procedures and reviewer bias.Keywords: research funding, peer review, impact metrics, validity, grant GRANT PEER REVIEW AND IMPACT ASSESSMENTMost would generally agree the purpose of biomedical research is to advance knowledge for societal benefit, with the hope of favorably impacting disease outcomes and improving global health. Indeed, the National Institutes of Health (NIH ), the world's largest funder of biomedical research, characterizes their mission as to "seek fundamental knowledge about the nature and Gallo and GlissonReview Validity via Impact Measures behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability" (NIH, 2017). To help select which research projects to fund to achieve...

show abstract

Peer review for improving the quality of grant applications

Demicheli¹,

Pietrantonj²

2003

Cochrane Database of Systematic Reviews

View full text Add to dashboard Cite

Untitled

Cited by 17 publications

References 16 publications

The impact of video abstract on citation counts: evidence from a retrospective cohort study of New Journal of Physics

The impact of video abstract on citation counts: evidence from a retrospective cohort study of New Journal of Physics

External Tests of Peer Review Validity Via Impact Measures

Peer review for improving the quality of grant applications

Contact Info

Product

Resources

About