Comparable Evaluation of Contemporary Corpus-Based and Knowledge-Based Semantic Similarity Measures of Short Texts

Abstract

This paper presents methods for measuring the semantic similarity of texts, where we evaluated different approaches based on existing similarity measures. On one side word similarity was calculated by processing large text corpuses and on the other, commonsense knowledgebase was used. Given that a large fraction of the information available today, on the Web and elsewhere, consists of short text snippets (e.g. abstracts of scientifi c documents, image captions or product descriptions), where commonsense knowledge has an important role, in this paper we focus on computing the similarity between two sentences or two short paragraphs by extending existing measures with information from the ConceptNet knowledgebase. On the other hand, an extensive research has been done in the fi eld of corpus-based semantic similarity, so we also evaluated existing solutions by imposing some modifi cations. Through experiments performed on a paraphrase data set, we demonstrate that some of proposed approaches can improve the semantic similarity measurement of short text.

Evaluation and classification of syntax usage in determining short-text semantic similarity

“…The Islam and Inkpen method [10] is close to it, although the inclusion of common-word order information into its similarity score actually leads to slightly worse results. Even though the statistical approach proposed by Furlan et al [11] is quite similar to the one in [10], the corpus used for training the COALS algorithm was rather small, which is why their results are, unsurprisingly, worse.…”

“…The performance of the SyMSS model of Oliva et al [19] increases if weighting according to semantic roles is applied, with verbs carrying the greatest weight, subjects and objects a somewhat smaller one, while adverbial complements and other roles are assigned even lower values. The topological approach of Furlan et al [11] reaches maximal accuracy levels by giving the verbs a weight four times greater than the one used for subjects and objects. Nevertheless, this method performs poorly in comparison to the statistical algorithm presented in the same paper because many subjects and objects consist of proper nouns which cannot be found in the ConceptNet knowledge base, effectively rendering those constituents irrelevant in the calculation of the similarity score.…”

“…They experimented with basic syntactic information in the form of the common-word order in the two texts. Furlan et al [11] considered a modified version of the Islam and Inkpen approach in which the COALS (Correlated Occurrence Analogue to Lexical Semantics) algorithm, trained on a corpus of article abstracts from the English-language Wikipedia, is used instead of SOC-PMI.…”

“…Furlan et al [11] proposed an algorithm which relies on the ConceptNet semantic network as its knowledge base. They employed an SRL mechanism able to extract subjectverb-object tuples from a given sentence.…”

A Survey and Evaluation of State-of-the-Art Intelligent Question Routing Systems

“…This method, besides the semantic word similarity measure, incorporates the string similarity measure, so it performs better with typos, evolving hotwords or different forms of infrequent proper nouns. 22…”