Citation function, polarity and influence classification

Abstract

Current methods for assessing the impact of authors and scientific media employ tools such as H-Index, Co-Citation and PageRank. These tools are primarily based on citation counting, which considers all citations to be equal. This type of methods can produce perverse incentives to publish controversial or incomplete papers, as mixed or negative reviews often generate larger citation counts and better indexes, regardless of whether the citations were critical or exerted minimal influence on the citing document. Passing citations that are employed to establish background, which do not have a real impact on the citing paper, are common in scientific literature. However, these citations have equal weight in impact evaluations. Notable researchers have emphasized the need to correct this situation by developing estimation methods that consider the different roles of quotations in citing papers. To accomplish this type of evaluation, a context citation analysis should be applied to determine the nature of the citations. We propose that citations should be categorized using four dimensions – FUNCTION, POLARITY, ASPECTS and INFLUENCE – as these dimensions provide adequate information that can be employed toward the generation of a qualitative method to measure the impact of a given publication in a citing paper. In this paper, we used interchangeably the words influence and impact. We present a method for obtaining this information using our proposed classification scheme and manually annotated corpus, which is marked with meaningful keywords and labels to help identify the characteristics or properties that constitute what we call ASPECTS. We develop a classification scheme which considers purpose definition shared by previous works. Our contribution is to abstract purpose classes from several other schemes and divide a complex structure in more manageable parts, to attain a simple system that combines low granularity dimensions but nevertheless produces a fine-grained classification. For annotators, the classification process is simple because in a first step, the coders distinguish only four primary classes, and in a second pass, they add the information contained in ASPECTS keyword and labels to obtain the more specific functions. This way, we gain a high granularity labeling that gives enough information about the citations to characterize and classify them, and we achieve this detailed coding with a straightforward process where the level of human error could be minimized.

What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018

“…Citation polarity determines the attitudes of the citing authors toward the cited paper, which could be favorable (positive), unfavorable (negative) or neither of these (neutral). However, citation purposes refer to citers' reasons and motivation in citing documents (Hernández-Alvarez et al, 2017;Jha et al, 2017).…”

“…For example, "use" can be seen in several studies Hernández-Alvarez et al, 2017;Jha et al, 2017) which happens to have a high frequency compared to other functions. "Use" has also been found to be among the most frequent verbs in different articles sections types (Bertin & Atanassova, 2014).…”

“…Therefore, through conventional citation analysis, we are unable to identify the specific contribution of a given work to the citing work (Anderson, 2006). Jha et al (2017) noted that a more robust measure of citations is to use the citation context to provide additional information about how a cited paper has been used in the citing paper (Hernández-Alvarez, Gomez Soriano, & Martínez-Barco, 2017). In other words, to understand citation impact, an extended form of citation analysis has been used by researchers, which is known as citation content/context analyses (Hernández -Alvarez & Gomez, 2015).…”

On the Possibilities of Evaluating Properties of Scientific Documents on the Basis of their Citations Count (or again: What Property is Reflected by Citations Count par excellence, after all?). Part 1: Value

“…For example, R. Jha et al (2017) indicated that the most frequent citation function was "use" (17.7%), and that the accuracy of their proposed classifier in recognizing "use" (60%) was higher than that of other functions. "Use" was found to be mentioned quite frequently in most sections, specifically methods and introduction sections"… But on the other hand, I. Tahamtan & L. Bornmann (2019) stated that "low precision of the classification scheme in detecting citation functions" was mentioned in one of the works under analysis; while, according to another cited paper, "the issue with the studies that use automated data processing is that the classification schemes they propose for identifying citation functions do not yield reliable results (Hernández-Alvarez et al, 2017;Tahamtan & Bornmann, 2019).…”

Examining influential factors for acknowledgements classification using supervised learning

“…Citation classification is the one approach of content-based citation analysis that aims to understand the influence of scientific literature on authors, such as the author's purpose regarding citing a paper or the function of a citation. However, identifying the key concepts in citation context automatically is complex and they are difficult to classify, because the differences among categories are too subtle and many categories make it hard to apply automatic methods, especially using linguistic markers [29][30]. Citation classification with three classes regularly refers to citation polarity [31].…”