Recognizing textual entailment: Rational, evaluation and approaches

Dagan, Ido; Dolan, Bill; Magnini, Bernardo; Roth, Dan

doi:10.1017/s1351324909990209

Cited by 128 publications

(119 citation statements)

References 14 publications

Supporting

Mentioning

112

Contrasting

Unclassified

Order By: Relevance

“…Another important line of related work is the Textual Entailment (TE) framework (Dagan et al, 2009;Glickman et al, 2005). A text fragment, T, is said to entail a textual hypothesis H if the truth of H can be most likely inferred from T. While TE can be an important component in a CDED approach, and perhaps vice versa, the tasks are quite different.…”

Section: Related Workmentioning

confidence: 99%

Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection

Rinott¹,

Dankin²,

Perez³

et al. 2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

158

171

View full text Add to dashboard Cite

Engaging in a debate with oneself or others to take decisions is an integral part of our day-today life. A debate on a topic (say, use of performance enhancing drugs) typically proceeds by one party making an assertion/claim (say, PEDs are bad for health) and then providing an evidence to support the claim (say, a 2006 study shows that PEDs have psychiatric side effects). In this work, we propose the task of automatically detecting such evidences from unstructured text that support a given claim. This task has many practical applications in decision support and persuasion enhancement in a wide range of domains. We first introduce an extensive benchmark data set tailored for this task, which allows training statistical models and assessing their performance. Then, we suggest a system architecture based on supervised learning to address the evidence detection task. Finally, promising experimental results are reported.

show abstract

Section: Related Workmentioning

confidence: 99%

Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection

Rinott¹,

Dankin²,

Perez³

et al. 2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

158

171

View full text Add to dashboard Cite

show abstract

“…3 Background 3.1 Natural Language Inference NLI, or RTE, is the task of inferring whether a natural language sentence (hypothesis) is entailed by another natural language sentence (premise) (Maccartney, 2009;Dagan et al, 2009;Dagan and Glickman, 2004).…”

Section: Behavior Analysismentioning

confidence: 99%

Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness

Sánchez¹,

Mitchell²,

Riedel³

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

Natural Language Inference is a challenging task that has received substantial attention, and state-of-the-art models now achieve impressive test set performance in the form of accuracy scores. Here, we go beyond this single evaluation metric to examine robustness to semantically-valid alterations to the input data. We identify three factors -insensitivity, polarity and unseen pairs -and compare their impact on three SNLI models under a variety of conditions. Our results demonstrate a number of strengths and weaknesses in the models' ability to generalise to new in-domain instances. In particular, while strong performance is possible on unseen hypernyms, unseen antonyms are more challenging for all the models. More generally, the models suffer from an insensitivity to certain small but semantically significant alterations, and are also often influenced by simple statistical correlations between words and training labels. Overall, we show that evaluations of NLI models can benefit from studying the influence of factors intrinsic to the models or found in the dataset used.

show abstract

“…The RTE can be seen as a binary classification task where the entailment relations are the classes, and the RTE benchmark datasets are used to train a classifier [8].…”

Section: Proposed Methodsmentioning

confidence: 99%

Recognizing Textual Entailment with a Semantic Edit Distance Metric

Rios

Gelbukh

2012

2012 11th Mexican International Conference on Artificial Intelligence

View full text Add to dashboard Cite

Abstract-We present a Recognizing Textual Entailment (RTE) system based on different similarity metrics. The metrics used are string-based metrics and the Semantic Edit Distance Metric, which is proposed in this paper to address limitations of known semantic-based metrics and to support the decisions made by a simple method based on lexical similarity metrics. We add the scores of the metrics as features for a machine learning algorithm. The performance of our system is comparable with the average performance of the Recognizing Textual Entailment Challenges, though lower than that of the state-ofthe-art methods.

show abstract

Recognizing textual entailment: Rational, evaluation and approaches

Cited by 128 publications

References 14 publications

Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection

Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection

Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness

Recognizing Textual Entailment with a Semantic Edit Distance Metric

Contact Info

Product

Resources

About