Granularity of algorithmically constructed publication-level classifications of research publications: Identification of topics

There are many different relatedness measures, based for instance on citation relations or textual similarity, that can be used to cluster scientific publications. We propose a principled methodology for evaluating the accuracy of clustering solutions obtained using these relatedness measures. We formally show that the proposed methodology has an important consistency property. The empirical analyses that we present are based on publications in the fields of cell biology, condensed matter physics, and economics. Using the BM25 text-based relatedness measure as evaluation criterion, we find that bibliographic coupling relations yield more accurate clustering solutions than direct citation relations and co-citation relations. The so-called extended direct citation approach performs similarly to or slightly better than bibliographic coupling in terms of the accuracy of the resulting clustering solutions.The other way around, using a citation-based relatedness measure as evaluation criterion, BM25 turns out to yield more accurate clustering solutions than other text-based relatedness measures.

Section: Quality Function For Clustering Publicationsmentioning

confidence: 99%

A principled methodology for comparing relatedness measures for clustering publications

Waltman

Boyack

Colavizza

2020

“…Further advances in these techniques were made by Waltman and van Eck (2012), who used direct citations with the minimum number of publications per cluster and a resolution parameter to come up with a 3-level classification. Their work has been further advanced by creating a number of algorithmic classifications at different levels of granularity (Ruiz-Castillo & Waltman, 2015) and searches for the optimal resolution parameter for the level of topics (Sjögårde & Ahlgren, 2018). In addition, since these methods are based on clustering algorithms, and it has long been argued that the resulting classifications are not algorithmneutral (Leydesdorff, 1987), some studies addressed how different algorithms affect resulting classifications (Šubelj, van Eck, & Waltman, 2016).…”

Section: Introductionmentioning

confidence: 99%

Practical method to reclassify Web of Science articles into unique subject categories and broad disciplines

Milojević

2020

Classification of bibliographic items into subjects and disciplines in large databases is essential for many quantitative science studies. The Web of Science classification of journals into ~250 subject categories, which has served as a basis for many studies, is known to have some fundamental problems and several practical limitations that may affect the results from such studies. Here we present an easily reproducible method to perform reclassification of the Web of Science into existing subject categories and into 14 broad areas. Our reclassification is at a level of articles, so it preserves disciplinary differences that may exist among individual articles published in the same journal. Reclassification also eliminates ambiguous (multiple) categories that are found for 50% of items, and assigns a discipline/field category to all articles that come from broad-coverage journals such as Nature and Science. The correctness of the assigned subject categories is evaluated manually and is found to be ~95%.

“…Duplicate IDs were removed before subsequent processing. Document clustering has previously been used to identify scientific topics (Sjögårde & Ahlgren, 2018;Waltman, Van Eck, & Noyons, 2010) and network mapping has been used to identify collaboration patterns (Xie, Ouyang, & Li, 2016), but the task here is different: to identify groups of articles created by a mostly common set of authors. A heuristic was used to identify publishing consortia.…”

Section: Methodsmentioning

confidence: 99%

Large publishing consortia produce higher citation impact research but coauthor contributions are hard to evaluate

Thelwall

2020

This paper introduces a simple agglomerative clustering method to identify large publishing consortia with at least 20 authors and 80% shared authorship between articles. Based on Scopus journal articles 1996-2018, under these criteria, nearly all (88%) of the large consortia published research with citation impact above the world average, with the exceptions being mainly the newer consortia for which average citation counts are unreliable. On average, consortium research had almost double (1.95) the world average citation impact on the log scale used (Mean Normalised Log Citation Score). At least partial alphabetical author ordering was the norm in most consortia. The 250 largest consortia were for nuclear physics and astronomy around expensive equipment, and for predominantly health-related issues in genomics, medicine, public health, microbiology and neuropsychology. For the health-related issues, except for the first and last few authors, authorship seem to primary indicate contributions to the shared project infrastructure necessary to gather the raw data. It is impossible for research evaluators to identify the contributions of individual authors in the huge alphabetical consortia of physics and astronomy, and problematic for the middle and end authors of health-related consortia. For small scale evaluations, authorship contribution statements could be used, when available.