Random walks for text semantic similarity

Ramage, Daniel; Rafferty, Anna N.; Manning, Christopher D.

doi:10.3115/1708124.1708131

Cited by 65 publications

(52 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Two well-known implementations of random walks on text graphs are TextRank (Mihalcea and Tarau 2004) and LexRank (Erkan and Radev 2004), variants and extensions of which have been applied to keyword detection, word sense disambiguation, text classification (Hassan and Banea 2006), summarisation (by extraction or query-biased Esuli andSebastiani 2007, or ontology-based Plaza et al 2008), novelty detection (Gamon 2006), lexical relatedness (Hughes and Ramage 2007), and semantic similarity estimation (Ramage et al 2009). To our knowledge, the only application of random walks term weights to IR has been our poster study of (Blanco and Lioma 2007), which we extend in this work.…”

Section: Text As Graphmentioning

confidence: 99%

Graph-based term weighting for information retrieval

Blanco

Lioma²

2011

Inf Retrieval

169

142

View full text Add to dashboard Cite

A standard approach to Information Retrieval (IR) is to model text as a bag of words. Alternatively, text can be modelled as a graph, whose vertices represent words, and whose edges represent relations between the words, defined on the basis of any meaningful statistical or linguistic relation. Given such a text graph, graph theoretic computations can be applied to measure various properties of the graph, and hence of the text. This work explores the usefulness of such graph-based text representations for IR. Specifically, we propose a principled graph-theoretic approach of (1) computing term weights and (2) integrating discourse aspects into retrieval. Given a text graph, whose vertices denote terms linked by co-occurrence and grammatical modification, we use graph ranking computations (e.g. PageRank Page et al. in The pagerank citation ranking: Bringing order to the Web. Technical report, Stanford Digital Library Technologies Project, 1998) to derive weights for each vertex, i.e. term weights, which we use to rank documents against queries. We reason that our graph-based term weights do not necessarily need to be normalised by document length (unlike existing term weights) because they are already scaled by their graph-ranking computation. This is a departure from existing IR ranking functions, and we experimentally show that it performs comparably to a tuned ranking baseline, such as BM25 (Robertson et al. in NIST Special Publication 500-236: TREC-4, 1995). In addition, we integrate into ranking graph properties, such as the average path length, or clustering coefficient, which represent different aspects of the topology of the graph, and by extension of the document represented as a graph. Integrating such properties into ranking allows us to consider issues such as discourse coherence, flow and density during retrieval. We experimentally show that this type of ranking performs comparably to BM25, and can even outperform it, across different TREC (Voorhees and Harman in TREC: Experiment and evaluation in information retrieval, MIT Press, 2005) datasets and evaluation measures.

show abstract

Section: Text As Graphmentioning

confidence: 99%

Graph-based term weighting for information retrieval

Blanco

Lioma²

2011

Inf Retrieval

169

142

View full text Add to dashboard Cite

show abstract

“…On similar word networks, work has also been done on understanding lexical network properties (Ferrer i Cancho and Solé 2001), or extracting words that follow certain semantic relations such as synonymy (Weale, Brew and Fosler-Lussier 2009). A significant amount of effort has also been put into the measurement of semantic distance using path-based algorithms on semantic networks (Lin 1998) or randomwalks (Ramage, Rafferty and Manning 2009). These random-walk algorithms have been successfully applied to other problems in semantics, such as word available at https://www.cambridge.org/core/terms.…”

Section: Graphs and Natural Language Processingmentioning

confidence: 99%

Word from the editors

2015

View full text Add to dashboard Cite

Graphs and natural language processingGraph structures naturally model connections. In natural language processing (NLP) connections are ubiquitous, on anything between small and web scale. We find them between words -as grammatical, collocation or semantic relations -contributing to the overall meaning, and maintaining the cohesive structure of the text and the discourse unity. We find them between concepts in ontologies or other knowledge repositories -since the early ages of artificial intelligence, associative or semantic networks have been proposed and used as knowledge stores, because they naturally capture the language units and relations between them, and allow for a variety of inference and reasoning processes, simulating some of the functionalities of the human mind. We find them between complete texts or web pages, and between entities in a social network, where they model relations at the web scale. Beyond the more often encountered 'regular' graphs, hypergraphs have also appeared in our field to model relations between more than two units.Graphs have been rigorously studied, both mathematically and computationally, providing a well-developed theoretical and practical base to the many natural language processing problems that map into this framework.In syntax, part-of-speech tagging was tackled using graph clustering (Biemann 2006) and dependency parsing using minimum spanning trees (McDonald et al. 2005). Related to parsing is the task of prepositional phrase attachment, which found interesting solutions in semi-supervised based learning (Toutanova, Manning and Ng 2004). Min-cut algorithms have also been used in text processing applications, for instance for the problem of coreference (Nicolae and Nicolae 2006).In semantics, graphs have been used to construct semantic classes (Widdows and Dorow 2002) through networks of words built from very large corpora. On similar word networks, work has also been done on understanding lexical network properties (Ferrer i Cancho and Solé 2001), or extracting words that follow certain semantic relations such as synonymy (Weale, Brew and Fosler-Lussier 2009). A significant amount of effort has also been put into the measurement of semantic distance using path-based algorithms on semantic networks (Lin 1998) There are also a number of natural language processing applications that found successful solutions in the use of graph-based methods. These include text summarization ( Overview of the issueThe four papers in the current special issue each showcase and exploit a different aspect/facet of graphs for different tasks in natural language processing.Kotlerman et al. present a novel twist on the graphs as a framework for the organization of knowledge. Textual Entailment Graphs expand the notion of the ontology as a network of concepts to larger text units that convey complex information. The entailment relation replaces the is-a relation from ontologies, and text fragments replace concepts, to obtain a structure that organizes text units by subsumption of the conv...

show abstract

“…Several contributions have also been proposed to compare larger units of language such as pairs of sentences or texts, e.g. (Corley and Mihalcea, 2005;Yu et al, 2006;Hughes and Ramage, 2007;Ramage et al, 2009;Buscaldi et al, 2013). However, most of these latter measures are extensions of measures which have been defined for comparing words, or rely on approaches which are also used to compare words, e.g.…”

Section: From Text Analysis To Semantic Measuresmentioning

confidence: 99%

“…Examples of measures based on random walk techniques are defined and discussed in (Muller et al, 2006;Hughes and Ramage, 2007;Ramage et al, 2009;Fouss et al, 2007;Garla and Brandt, 2012).…”

Section: Random Walk Approachesmentioning

confidence: 99%

Semantic Similarity from Natural Language and Ontology Analysis

Harispe

Ranwez

Janaqi

et al. 2015

Synthesis Lectures on Human Language Technologies

166

102

View full text Add to dashboard Cite

Artificial Intelligence federates numerous scientific fields in the aim of developing machines able to assist human operators performing complex treatments -most of which demand high cognitive skills (e.g. learning or decision processes). Central to this quest is to give machines the ability to estimate the likeness or similarity between things in the way human beings estimate the similarity between stimuli.In this context, this book focuses on semantic measures: approaches designed for comparing semantic entities such as units of language, e.g. words, sentences, or concepts and instances defined into knowledge bases. The aim of these measures is to assess the similarity or relatedness of such semantic entities by taking into account their semantics, i.e. their meaning -intuitively, the words tea and coffee, which both refer to stimulating beverage, will be estimated to be more semantically similar than the words toffee (confection) and coffee, despite that the last pair has a higher syntactic similarity. The two state-of-theart approaches for estimating and quantifying semantic similarities/relatedness of semantic entities are presented in detail: the first one relies on corpora analysis and is based on Natural Language Processing techniques and semantic models while the second is based on more or less formal, computer-readable and workable forms of knowledge such as semantic networks, thesaurus or ontologies.Semantic measures are widely used today to compare units of language, concepts, instances, or even resources indexed by them (e.g., documents, genes). They are central elements of a large variety of Natural Language Processing applications and knowledge-based treatments, and have therefore naturally been subject to intensive and interdisciplinary research efforts during last decades. Beyond a simple inventory and categorization of existing measures, the aim of this monograph is to convey novices as well as researchers of these domains towards a better understanding of semantic similarity estimation and more generally semantic measures. To this end, we propose an in-depth characterisation of existing proposals by discussing their features, the assumptions on which they are based and empirical results regarding their performance in particular applications. By answering these questions and by providing a detailed discussion on the foundations of semantic measures, our aim is to give the reader key knowledge required to: (i) select the more relevant methods according to a particular usage context, (ii) understand the challenges offered to this field of study, (iii) distinguish room of improvements for state-of-the-art approaches and (iv) stimulate creativity towards the development of new approaches. In this aim, several definitions, theoretical and practical details, as well as concrete applications are presented.keywords: Semantic similarity, semantic relatedness, semantic measures, distributional measures, domain ontology, knowledge-based semantic measure.iii iv

show abstract

Random walks for text semantic similarity

Cited by 65 publications

References 25 publications

Graph-based term weighting for information retrieval

Graph-based term weighting for information retrieval

Word from the editors

Semantic Similarity from Natural Language and Ontology Analysis

Contact Info

Product

Resources

About