Measuring novelty in science with word embedding

Abstract

Novelty is a core value in science, and a reliable measurement of novelty is crucial. This study proposes a new approach of measuring the novelty of scientific articles based on both citation data and text data. The proposed approach considers an article to be novel if it cites a combination of semantically distant references. To this end, we first assign a word embedding–a vector representation of each vocabulary–to each cited reference on the basis of text information included in the reference. With these vectors, a distance between every pair of references is computed. Finally, the novelty of a focal document is evaluated by summarizing the distances between all references. The approach draws on limited text information (the titles of references) and publicly shared library for word embeddings, which minimizes the requirement of data access and computational cost. We share the code, with which one can compute the novelty score of a document of interest only by having the focal document’s reference list. We validate the proposed measure through three exercises. First, we confirm that word embeddings can be used to quantify semantic distances between documents by comparing with an established bibliometric distance measure. Second, we confirm the criterion-related validity of the proposed novelty measure with self-reported novelty scores collected from a questionnaire survey. Finally, as novelty is known to be correlated with future citation impact, we confirm that the proposed measure can predict future citation.

An integrative model of new product evaluation: A systematic investigation of perceived novelty and product evaluation in the movie industry

“…In the literature, another concept related to perceived novelty is product novelty, which refers to the degree to which a new product creates a novel product category or integrates extant product categories in novel ways [12,[44][45][46][47][48]. There are two assumptions underlying this definition-(1) a novel product creates a completely new product category because the current category cannot categorize such a novel product, or (2) a novel product combines extant product categories in a unique way.…”

“…There are two assumptions underlying this definition-(1) a novel product creates a completely new product category because the current category cannot categorize such a novel product, or (2) a novel product combines extant product categories in a unique way. Past researchers [12,[44][45][46][47][48] conceptualized and operationalized product novelty relying on the second assumption (i.e., novel combinations of extant product categories) because in reality, it is extremely rare for novelty to be so radical that a new product creates a completely new product category. For example, researchers have measured product novelty by considering whether a new financial report uses novel combinations of extant repertoires (or, in their words, portfolios of framing [46]) and how much researchers use novel combinations of upper-level keywords in research proposals to describe their new work (see, for example [44,45,47]).…”

Digital Document Analysis Using Weighted Score Convolutional Neural Network and Arc Factored Graph Based Dependency Parsing

“…The novelty of a focal document has been estimated by narrating the distances [19] between all references based on the titles. This ensures a mechanism for future citation also.…”

Automated scholarly paper review: Technologies and challenges

“…This can be the basis of distinguishing papers with originality from those without. Shibayama et al (2021) designed a more integrated method, making use of both the citations and content of a paper. In this method, the semantic distance of references is quantified to determine the novelty of a given paper.…”