Counting methods, country rank changes, and counting inflation in the assessment of national research productivity and impact

Abstract

The counting of papers and citations is fundamental to the assessment of research productivity and impact. In an age of increasing scientific collaboration across national borders, the counting of papers produced by collaboration between multiple countries, and citations of such papers, raises concerns in country-level research evaluation. In this study, we compared the number counts and country ranks resulting from five different counting methods. We also observed inflation depending on the method used. Using the 1989 to 2008 physics papers indexed in ISI's Web of Science as our sample, we analyzed the counting results in terms of paper count (research productivity) as well as citation count and citation-paper ratio (CP ratio) based evaluation (research impact). The results show that at the countrylevel assessment, the selection of counting method had only minor influence on the number counts and country rankings in each assessment. However, the influences of counting methods varied between paper count, citation count, and CP ratio based evaluation. The findings also suggest that the popular counting method (whole counting) that gives each collaborating country one full credit may not be the best counting method. Straight counting that accredits only the first or the corresponding author or fractional counting that accredits each collaborator with partial and weighted credit might be the better choices.

Field-normalized citation impact indicators and the choice of an appropriate counting method

“…Recent work on the comparison of counting methods is reported by Huang, Lin, and Chen (2011) and Lin, Huang, and Chen (2013). These authors take into account both publication output and citation impact.…”

A study of research collaboration in the pre‐web and post‐web stages: A coauthorship analysis of the information systems discipline

“…We retrieved the names of the first authors and corresponding authors for these 27,043 articles, for the following reasons. First, as evidenced by Huang, Lin, and Chen (), first author or corresponding author is a rather reliable and cost‐effective choice in assessing research productivity as the method of straight counting that accredits only first or corresponding author and the fractional counting that equally accredits each author generate highly similar result. Second, the order of authors for IS research is important, and IS institutions typically give more credit to first authors (Peffers & Hui, ).…”

Independent research of China in Science Citation Index Expanded during 1980–2011

“…Corresponding author was the one who contributed the most to the initial conception and supervision (Wren et al, 2007). Straight counting that accredits only the first or the corresponding author fractional counting that accredits each collaborator with partial and weighted credit might be the better choices (Huang, Lin, & Chen, 2011). Y-index (j, Â) (Ho, 2012a) which takes the first author articles and corresponding author articles into consideration could provide more fair information for evaluation of Chinese institutions and authors.…”