Interpretable semantic textual similarity: Finding and explaining differences between sentences

López-Gazpio, Iñigo; Maritxalar, Montse; González-Agirre, Aitor; Rigau, Germán; Uria, Larraitz; Agirre, Eneko

doi:10.1016/j.knosys.2016.12.013

Cited by 39 publications

(12 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, by detecting synsets that are frequently involved in problems classified as incompatible. Finally, we plan to evaluate knowledge resources such as ontology domains [64] and the Multilingual Central Repository (MCR) [33], and to check the utility of Adimen-SUMO v2.6 in Natural Language Processing (NLP) tasks that involve reasoning on commonsense knowledge [18], such as Recognizing Textual Entailment (RTE) [1], [19], [21], Natural Language Inference (NLI) [20] or Interpretable Semantic Textual Similarity (iSTS) [45].…”

Section: Discussionmentioning

confidence: 99%

A Framework for the Evaluation of SUMO-Based Ontologies Using WordNet

2019

View full text Add to dashboard Cite

This paper offers a new practical approach toward automated commonsense reasoning with first-order logic (FOL) SUMO-based ontologies. We propose a new black-box evaluation framework for SUMO-based ontologies, which exploits the world knowledge encoded in WordNet and its mapping into SUMO. Our proposal consists of both a novel semi-automatic method for the creation of a large set of commonsense problems and a new procedure that enables its automatic evaluation by using automated theorem provers (ATPs). The application of our method enables the creation of a very large benchmark consisting of more than 15 000 problems from a small set of manually built question patterns that exploit the WordNet semantic relations. By means of the resulting benchmark, we successfully evaluate the competency of different translations of SUMO into FOL and the performance of various state-of-the-art FOL ATPs according to several quality criteria. A general analysis of our experimental results demonstrates that the proposed commonsense problems are heterogeneous and non-trivial. Furthermore, a fine-grained analysis of the experimental results obtained for a sample of our benchmark enables the detection of some mapping errors and some discrepancies between the knowledge of WordNet and SUMO. The evaluation benchmark and all the resources that have been used and developed during this work are released in a single package. INDEX TERMS Ontology evaluation, automated reasoning, commonsense knowledge, WordNet, SUMO.

show abstract

Section: Discussionmentioning

confidence: 99%

A Framework for the Evaluation of SUMO-Based Ontologies Using WordNet

2019

View full text Add to dashboard Cite

show abstract

“…For example, by detecting synsets that are frequently involved in problems classified as incompatible. Finally, we plan to evaluate the knowledge in the Multilingual Central Repository (MCR) [23] and to check the utility of Adimen-SUMO v2.6 in Natural Language Processing (NLP) tasks that involve reasoning on commonsense knowledge [11], such as Recognizing Textual Entailment (RTE) [12,14,1], Natural Language Inference (NLI) [13] or Interpretable Semantic Textual Similarity (ISTS) [29].…”

Section: Discussionmentioning

confidence: 99%

Black-box Testing of First-Order Logic Ontologies Using WordNet

Àlvez¹,

Lucio²,

Rigau³

2017

Preprint

View full text Add to dashboard Cite

Artificial Intelligence aims to provide computer programs with commonsense knowledge to reason about our world. This paper offers a new practical approach towards automated commonsense reasoning with first-order logic (FOL) ontologies. We propose a new black-box testing methodology of FOL SUMO-based ontologies by exploiting WordNet and its mapping into SUMO. Our proposal includes a method for the (semi-)automatic creation of a very large benchmark of competency questions and a procedure for its automated evaluation by using automated theorem provers (ATPs). Applying different quality criteria, our testing proposal enables a successful evaluation of a) the competency of several translations of SUMO into FOL and b) the performance of various automated ATPs. Finally, we also provide a fine-grained and complete analysis of the commonsense reasoning competency of current FOL SUMO-based ontologies.

show abstract

“…The core STS task was extended to include the interpretability of similarity scores (Agirre et al, 2015;Lopez-Gazpio et al, 2017). The goal of the interpretable STS is to provide reasoning behind the assigned similarity scores by identifying the alignment between pairs of segments across the two sentences, assigning to each alignment a relation type and a similarity score.…”

Section: Related Workmentioning

confidence: 99%

A resource-light method for cross-lingual semantic textual similarity

Glavaš

Franco-Salvador

Ponzetto

et al. 2018

Knowledge-Based Systems

View full text Add to dashboard Cite

Recognizing semantically similar sentences or paragraphs across languages is beneficial for many tasks, ranging from cross-lingual information retrieval and plagiarism detection to machine translation. Recently proposed methods for predicting cross-lingual semantic similarity of short texts, however, make use of tools and resources (e.g., machine translation systems, syntactic parsers or named entity recognition) that for many languages (or language pairs) do not exist. In contrast, we propose an unsupervised and a very resource-light approach for measuring semantic similarity between texts in different languages. To operate in the bilingual (or multilingual) space, we project continuous word vectors (i.e., word embeddings) from one language to the vector space of the other language via the linear translation model. We then align words according to the similarity of their vectors in the bilingual embedding space and investigate different unsupervised measures of semantic similarity exploiting bilingual embeddings and word alignments. Requiring only a limited-size set of word translation pairs between the languages, the proposed approach is applicable to virtually any pair of languages for which there exists a sufficiently large corpus, required to learn monolingual word embeddings. Experimental results on three different datasets for measuring semantic textual similarity show that our simple resource-light approach * Corresponding author Email addresses: goran@informatik.uni-mannheim.de (Goran Glavaš), mfranco@prhlt.upv.es (Marc Franco-Salvador), simone@informatik.uni-mannheim.de (Simone P. Ponzetto), prosso@dsic.upv.es (Paolo Rosso) reaches performance close to that of supervised and resource-intensive methods, displaying stability across different language pairs. Furthermore, we evaluate the proposed method on two extrinsic tasks, namely extraction of parallel sentences from comparable corpora and cross-lingual plagiarism detection, and show that it yields performance comparable to those of complex resource-intensive state-of-the-art models for the respective tasks.

show abstract

Interpretable semantic textual similarity: Finding and explaining differences between sentences

Cited by 39 publications

References 24 publications

A Framework for the Evaluation of SUMO-Based Ontologies Using WordNet

A Framework for the Evaluation of SUMO-Based Ontologies Using WordNet

Black-box Testing of First-Order Logic Ontologies Using WordNet

A resource-light method for cross-lingual semantic textual similarity

Contact Info

Product

Resources

About