Evaluating popularity data for relevance ranking in library information systems

Abstract

In this poster, we present our work in progress to develop a relevance model for library information systems, which takes non-textual factors into account. Here we focus on popularity data like citation or usage data. These data contain various biases that need to be corrected so as not to degrade the performance of the relevance model. Further, the different data might be to some extent incommensurable. We make use of the Characteristic Scores and Scales method to achieve two goals: first, remove biases from the raw data, and second, establish a common scale for the different data to support weighing the data against each other.

Altmetrics for Digital Libraries. Concepts, Applications, Evaluation and Recommendations

“…The authors used the Characteristic Score and Scales (CSS) method to correct the citation data and usage data biases and to classify articles into "poorly cited," "fairly cited," "remarkably cited," and "outstandingly cited" groups. This method is confirmed as highly promising by producing valuable benefits to the users' needs (Plassmeier et al, 2015). However, the CSS worked well for normalizing citation data but not for usage data, suggesting that future studies are needed to explore other normalization methods for the better performance of the relevance model.…”

“…However, a project known as LibRank 32 , which is being developed within the EconBiz environment and data and funded by the German Research Foundation (DFG), started to investigate other relevance ranking methods for better performance in EconBiz. One of the experiments that LibRank considered is to rank search results based on popularity factors (i.e., citations; Plassmeier et al, 2015). Figure 1.5: Year-wise journal indexing in ZBW.…”

“…Figure 1.5: Year-wise journal indexing in ZBW. Plassmeier et al (2015) studied the relevance of search results based on non-textual factors or popularity-based factors that focus on citation counts of articles, author metrics, and usage data rather than text statistics (e.g., the position and the frequency of the search term in the article or publication dates). The authors used the Characteristic Score and Scales (CSS) method to correct the citation data and usage data biases and to classify articles into "poorly cited," "fairly cited," "remarkably cited," and "outstandingly cited" groups.…”

“…Journals represented as A+ are ranked as highly important according to JOURQUAL2, and all of them are covered by EconBiz, Scopus, and Web Science. EconBiz, however, does not collect or provide citation indexes to the journal articles but explores citation counts (derived from CitEc 39 a RePEc service for bibliometrics) as sources for retrieval purposes, that is, a better ranking of search results (Plassmeier et al, 2015).…”

“…During the last decade, different information retrieval and ranking strategies for library portals were suggested, developed, and validated to rapidly show relevant articles throughout the users' search process and therefore minimize the display of irrelevant articles (Plassmeier et al, 2015;Bian et al, 2017;Damarell et al, 2019).…”