Automated Research Impact Assessment: a new bibliometrics approach

Drew, Christina H.; Pettibone, Kristianna G.; Finch, Fallis Owen; Giles, Douglas; Jordan, Paul H.

doi:10.1007/s11192-015-1828-7

Cited by 23 publications

(15 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, five unique definitions were identified in this review that fell outside the typologies constructed [ 61 , 74 – 77 ]. These definitions all drew distinctions between the ideas of impacts, outputs and/or outcomes, and shared some of the features of the aforementioned definitions.…”

Section: Discussionmentioning

confidence: 99%

How do we define the policy impact of public health research? A systematic review

Alla

Hall

Whiteford

et al. 2017

Health Res Policy Sys

View full text Add to dashboard Cite

BackgroundIn order to understand and measure the policy impact of research we need a definition of research impact that is suited to the task. This article systematically reviewed both peer-reviewed and grey literature for definitions of research impact to develop a definition of research impact that can be used to investigate how public health research influences policy.MethodKeyword searches of the electronic databases Web of Science, ProQuest, PubMed, EMBASE, CINAHL, Informit, PsycINFO, The Cochrane Database of Systematic Reviews and Google Scholar were conducted between August 2015 and April 2016. Keywords included ‘definition’ and ‘policy’ and ‘research impact’ or ‘research evidence’. The search terms ‘health’, public health’ or ‘mental health’ and ‘knowledge transfer’ or ‘research translation’ were used to focus the search on relevant health discipline approaches. Studies included in the review described processes, theories or frameworks associated with public health, health services or mental health policy.ResultsWe identified 108 definitions in 83 publications. The key findings were that literature on research impact is growing, but only 23% of peer-reviewed publications on the topic explicitly defined the term and that the majority (76%) of definitions were derived from research organisations and funding institutions. We identified four main types of definition, namely (1) definitions that conceptualise research impacts in terms of positive changes or effects that evidence can bring about when transferred into policies (example Research Excellence Framework definition), (2) definitions that interpret research impacts as measurable outcomes (Research Councils UK), and (3) bibliometric and (4) use-based definitions. We identified four constructs underpinning these definitions that related to concepts of contribution, change, avenues and levels of impact.ConclusionThe dominance of bureaucratic definitions, the tendency to discuss but not define the concept of research impact, and the heterogeneity of definitions confirm the need for conceptual clarity in this area. We propose a working definition of research impact that can be used in a range of health policy contexts.Electronic supplementary materialThe online version of this article (doi:10.1186/s12961-017-0247-z) contains supplementary material, which is available to authorized users.

show abstract

Section: Discussionmentioning

confidence: 99%

How do we define the policy impact of public health research? A systematic review

Alla

Hall

Whiteford

et al. 2017

Health Res Policy Sys

View full text Add to dashboard Cite

show abstract

“…The main features of the papers were described and historical trends assessed with regard to number of papers, journals publishing on the subject, quality of the publications, topics investigated and the most studied regions and states of Brazil. The evaluation of scientific production in natural sciences has been the topic of discussion in recent years Nabout et al, 2012;Borges et al, 2015;Nabout et al, 2015;Vaz et al, 2015;Santos et al, 2017), and has become an important tool for identifying trends and biases in the scientific production of a given area of study (Drew et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

30 years of research on insect galls in Brazil: a scientometric review

Araújo

2018

Pap. Avulsos Zool.

View full text Add to dashboard Cite

The first systematic studies on insect galls in Brazil date to the early 20th century, after which research on insect galls remained dormant in the country, with interest not reviving until the 1980s. The aim of this study was is to document historical trends in publications about insect galls in Brazil over the last 30 years. Papers about insect galls and galling species in Brazil published in peer-reviewed journals from 1988 to 2017 were compiled. A total of 1,378 papers were analyzed, of which 182 addressed insect galls in Brazil. The results showed that the number of publications on the subject has been increasing over the last 30 years, and especially in the last decade. The diversity of journals that published on the subject has also been increasing. The studies were concentrated on the following topics: ecology (94 papers), inventory (29) and taxonomy (27). Most of the insect gall inventories in Brazil took place in the Southeast Region (29 papers), followed by the Central-West and Northeast regions, with eight papers each. This study documents a trend toward increasing scientific production on insect galls in Brazil, but with significant geographical bias: the researchers involved are concentrated in the Southeast Brazilian region.

show abstract

“…A number of authors have provided thoughtful reviews of the limitations of bibliometric analyses (Texeira da Silva and Dobránszki 2018 ; Costas and Franssen 2018 ). These limitations include biases of bibliometric comparisons across fields or countries (Gingras and Khelfaoui 2018 ); a tendency to favor older researchers with more experience (Kwiek 2018 ) or wider networks (Ronda-Pupo and Katz 2018 ; Akbaritabar et al 2018 ); political motivations (Kostoff 1998 ); and a susceptibility to bias, such as self-citations (Thelwall 2018 ; Drew et al 2016 ). The as yet unresolved problems of multiple co-authorship (Bao and Zhai 2017 ; Ronda-Pupo and Katz 2018 ), and the fact that the annual publication capacity of top journals measured in pages varies widely across fields of science could also be mentioned in this respect.…”

Section: Introductionmentioning

confidence: 99%

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

et al. 2018

View full text Add to dashboard Cite

The Momentum program launched in 2009 provides funding of up to 1 million Euro to establish new, independent research groups at Hungarian academic institutions. Here, our aim was to determine factors associated with the scientific output of these research groups. Publication data were downloaded from the Hungarian Scientific Work Archive (www.mtmt.hu), impact factor data were obtained from Thomson Reuters (jcr.incites.thomsonreuters.com), and journal ranks were extracted from the Scimago Journal Rank database (www.scimagojr.com). Investigated input features for each grant holder include gender, degree, targeted category, international mobility, international grants, number of publications, total number of citations, H-index, best publications, impact factors in the last 2 years, and assessment scores provided by the experts. Evaluated performance indicators include cumulative impact factor, number of D1 publications, and number of first/last author D1 publications during the grant running time. Grant holders’ publication output increased by 23 and 52% for life sciences and material sciences researchers. Scientific performance was independent from gender, degree, international grants, category applied for, and citations received for the best pre-grant publication. Those with international mobility had significantly lower scientific output (yearly impact factor, number of D1 publications, number of first/last author publications). Scores received from grant review experts were independent from later publication activity. The strongest correlations were observed between scientific output and total number of citations, H-index, and impact factor in the last 2 years pre-grant. In summary, group leaders with a dynamic publication track record were able to attain the most additional momentum. Our results can help accelerate and improve future grant review processes.

show abstract

Automated Research Impact Assessment: a new bibliometrics approach

Cited by 23 publications

References 30 publications

How do we define the policy impact of public health research? A systematic review

How do we define the policy impact of public health research? A systematic review

30 years of research on insect galls in Brazil: a scientometric review

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

Contact Info

Product

Resources

About