Bibliometrics as a Performance Measurement Tool for Research Evaluation: The Case of Research Funded by the National Cancer Institute of Canada

Abstract

As bibliometric indicators are objective, reliable, and cost-effective measures of peer-reviewed research outputs, they are expected to play an increasingly important role in research assessment/management. Recently, a bibliometric approach was developed and integrated within the evaluation framework of research funded by the National Cancer Institute of Canada (NCIC). This approach helped address the following questions that were difficult to answer objectively using alternative methods such as program documentation review and key informant interviews: (a) Has the NCIC peer-review process selected outstanding Canadian scientists in cancer research? (b) Have the NCIC grants contributed to increasing the scientific performance of supported researchers? (c) How do the NCIC-supported researchers compare to their neighbors supported by the U.S. National Cancer Institute? Using the NCIC evaluation as a case study, this article demonstrates the usefulness of bibliometrics to address key evaluation questions and discusses its integration, along complementary indicators (e.g., peer ratings), in a practice-driven research evaluation continuum.

Mexican Breast Cancer Research Output, 2003-2012

“…Several papers have examined cancer research in individual countries: in Canada (Campbell et al, 2010), Japan (Ho et al, 2010), and Puerto Rico (Calo et al, 2010). Particularly, there is a study related to portuguese breast cancer research (Donato and De Oliveira, 2006).…”

A Bibliometrics Analysis of Canadian Electrical and Computer Engineering Institutions (1996-2006) Based on IEEE Journal Publications

“…Numerous bibliometric studies have been published in different disciplines such as, in cancer research (Campbell et al, 2010), fisheries (Science-Metrix Inc., 2011b), computer science/engineering (D. Katsaros, V. Matsoukas, & Y. Manolopoulos, 2008), software engineering (Garousi & Varma, 2010;Glass & Chen, 2002;2003;Ren & Taylor, 2007b), electrical engineering (Tsay, Jou, & Ma, 2000), across all engineering disciplines (Eckel, 2009), and obesity research (Canadian Institutes of Health Research, 2011). It is clear to observe that the number of bibliometric studies has been on the rise in recent years.…”

“…It is clear to observe that the number of bibliometric studies has been on the rise in recent years. Bibliometric studies also differ in covering different geographical areas/scales, e.g., world-wide bibliometric assessments, e.g., (Glass & Chen, 2002, 2003, or assessments only across a nation or between several nations, e.g., in Canada (Campbell et al, 2010;Garousi & Varma, 2010;Science-Metrix Inc., 2011a;2011b;2011c), the USA (Gaughan & Bozeman, 2002), China (Rong, Hua, & Hong, 2009) and Greece (Katsaros et al, 2008).…”

“…To prevent future debates on the results of our study, and to maximize transparency and replicability of our methods and results, we discuss the details of our methodology first before reporting the bibliometric results. In reviewing the large literature of the existing bibliometric studies (Campbell et al, 2010;Canadian Institutes of Health Research, 2011;Gaughan & Bozeman, 2002;Glass, 1995), it is somewhat surprising that most studies do not include details to assess the transparency and reproducibility of those studies. To address this issue, we have made all the effort to make our study 100% reproducible by discussing our methodology in detail and also by providing the entire data set we have used and generated for our analysis in an online webpage (Garousi, 2011).…”

Cientometria: a produção científica de Haity Moussatché

“…A cientometria, como suporte e orientação nas decisões relativas à gestão de ciência e tecnologia, floresceu a partir do final dos anos 1970, e cada vez mais os governos estão reconhecendo a importância da responsabilidade dos gastos públicos em pesquisa, o que tem aumentado a necessidade de equilibrar a avaliação mais tradicional de revisão por pares com métodos objetivos e quantitativos (Campbell et al, 2010).…”