The third PASCAL recognizing textual entailment challenge

Giampiccolo, Danilo; Magnini, Bernardo; Dagan, Ido; Dolan, Bill

doi:10.3115/1654536.1654538

Cited by 413 publications

(379 citation statements)

References 1 publication

Supporting

Mentioning

374

Contrasting

Unclassified

Order By: Relevance

“…Condoravdi et al first argued the importance of handling contradiction in text understanding [2]. As a kind of relation in the entailment recognition problem, contradiction has been studied continually in Recognizing Textual Entailment Challenges [13,[24][25][26][27][28][29]. In previous years, researchers undertook the task by resolving some of the contradiction phenomena [1,30], like negation, antonyms, data/number mismatch and different structure.…”

Section: Related Workmentioning

confidence: 99%

Contradiction Detection with Contradiction-Specific Word Embedding

Qin

Liu

2017

Algorithms

View full text Add to dashboard Cite

Contradiction detection is a task to recognize contradiction relations between a pair of sentences. Despite the effectiveness of traditional context-based word embedding learning algorithms in many natural language processing tasks, such algorithms are not powerful enough for contradiction detection. Contrasting words such as "overfull" and "empty" are mostly mapped into close vectors in such embedding space. To solve this problem, we develop a tailored neural network to learn contradiction-specific word embedding (CWE). The method can separate antonyms in the opposite ends of a spectrum. CWE is learned from a training corpus which is automatically generated from the paraphrase database, and is naturally applied as features to carry out contradiction detection in SemEval 2014 benchmark dataset. Experimental results show that CWE outperforms traditional context-based word embedding in contradiction detection. The proposed model for contradiction detection performs comparably with the top-performing system in accuracy of three-category classification and enhances the accuracy from 75.97% to 82.08% in the contradiction category.

show abstract

Section: Related Workmentioning

confidence: 99%

Contradiction Detection with Contradiction-Specific Word Embedding

Qin

Liu

2017

Algorithms

View full text Add to dashboard Cite

show abstract

“…Thus, we provided the hypothesis already built from the questions and answers [6] (see Figure 2). Then, the exercise was similar to the RTE Challenges [1] [2] [3], where systems must decide if there is entailment or not between the supporting text and the hypothesis. In this edition, on the contrary, we left open the problem of Automatic Hypothesis Generation for those systems based on Textual Entailment.…”

Section: Question Answering Trackmentioning

confidence: 99%

Overview of the Answer Validation Exercise 2008

Rodrigo

Peñas

Verdejo

2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The Answer Validation Exercise at the Cross Language Evaluation Forum is aimed at developing systems able to decide whether the answer of a Question Answering system is correct or not. We present here the exercise description, the changes in the evaluation methodology with respect to the first edition, and the results of this second edition (AVE 2007). The changes in the evaluation methodology had two objectives: the first one was to quantify the gain in performance when more sophisticated validation modules are introduced in QA systems. The second objective was to bring systems based on Textual Entailment to the Automatic Hypothesis Generation problem which is not part itself of the Recognising Textual Entailment (RTE) task but a need of the Answer Validation setting. 9 groups have participated with 16 runs in 4 different languages. Compared with the QA systems, the results show an evidence of the potential gain that more sophisticated AV modules introduce in the task of QA.

show abstract

“…The text and hypothesis pair is available there in the Italian language. Microsoft Bing translator API 5 for Bing translator (the file microsoft-translatorjava-api-0.4-jar-with-dependencies.jar) was used to translate the T and H text sentences into English.…”

Section: Pre-processingmentioning

confidence: 99%

“…There have been held seven Recognizing Textual Entailment (RTE) competitions: RTE-1 in 2005 [2], RTE-2 [3] in 2006, RTE-3 [4] in 2007, RTE-4 [5] in 2008, RTE-5 [6] in 2009, RTE-6 [7] in 2010, RTE-7 [8] in 2011. In 2010, Parser Training and Evaluation using Textual Entailment event [9] was organized in frame of SemEval-2.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recognizing Textual Entailment in Non-english Text via Automatic Translation into English

Neogi

Bandyopadhyay

Gelbukh

2013

Advances in Computational Intelligence

View full text Add to dashboard Cite

Abstract. We show that a task that typically involves rather deep semantic processing of text-being recognizing textual entailment our case study-can be successfully solved without any tools at all specific for the language of the texts on which the task is performed. Instead, we automatically translate the text into English using a standard machine translation system, and then perform all linguistic processing, including syntactic and semantic levels, using only English language linguistic tools. In this case study we use Italian annotated data. Textual entailment is a relation between two texts. To detect it, we use various measures, which allow us to make entailment decision in the two-way classification task (YES / NO). We set up various heuristics and measures for evaluating the entailment between two texts based on lexical relations. To make entailment judgments, the system applies named entity recognition module, chunking, partof-speech tagging, n-grams, and text similarity modules to both texts, all those modules being for English and not for Italian. Rules have been developed to perform the two-way entailment classification. Our system makes entailment judgments basing on the entailment scores for the text pairs. The system was evaluated on Italian textual entailment data sets: we trained our system on Italian development datasets using the WEKA machine learning toolset and tested it on Italian test data sets. The accuracy of our system on the development corpus is 0.525 and on the test corpus is 0.66, which is a good result given that no Italian-specific linguistic information was used.

show abstract

The third PASCAL recognizing textual entailment challenge

Cited by 413 publications

References 1 publication

Contradiction Detection with Contradiction-Specific Word Embedding

Contradiction Detection with Contradiction-Specific Word Embedding

Overview of the Answer Validation Exercise 2008

Recognizing Textual Entailment in Non-english Text via Automatic Translation into English

Contact Info

Product

Resources

About