What Syntax Can Contribute in the Entailment Task

Abstract

This paper describes the PASCAL Network of Excellence Recognising Textual Entailment (RTE) Challenge benchmark 1 . The RTE task is defined as recognizing, given two text fragments, whether the meaning of one text can be inferred (entailed) from the other. This applicationindependent task is suggested as capturing major inferences about the variability of semantic expression which are commonly needed across multiple applications. The Challenge has raised noticeable attention in the research community, attracting 17 submissions from diverse groups, suggesting the generic relevance of the task.

A machine learning approach to textual entailment recognition

“…However, as different linguistic levels can be adopted to represent T and H (lexical, syntactic, semantic), we here need to choose the most relevant, in order to better focus our study. For this purpose, we note that a large part of the entailment cases depend on the syntactic structure of T and H (Vanderwende and Dolan 2006). More specifically, grammar rules are most useful, as they can reduce data sparseness by generalizing word sequences expressed with the same syntax.…”

“…For this purpose, we use the gold standard of entailment examples provided by Vanderwende and Dolan ( 2006). In their study of the RTE1 dataset the authors discovered that 390 pairs out of the 800 of the test set can be classified using solely syntactic cues.…”

“…Experimental results of our model over the above dataset are reported in Table 6: the first column reports the feature space; the first row represents the classes of syntactic phenomena in Vanderwende and Dolan (2006). The second row shows the number of cases falling in each class according to the manual gold standard (note that examples can belong to more than one class when more than one transformation takes place); and all the other rows illustrate the accuracy of our different models when classifiers are trained on the RT E1 development set.…”

Discourse oriented summarization

“…In this chapter, the relative performance of each of the representation methods and alignment methods is measured. Vanderwende and Dolan (2005) analyzed the PASCAL-RTE test set to investigate what percentage of entailment relations could be recognized by using lexical-syntactic information. They concluded that a respectable 34 percent of the samples could be solved using syntax alone.…”

“…While Vanderwende and Dolan (2005) and Bar-Haim et al (2005) use manual annotations, this study seeks to compare performance and analyze the problems of using various representations and matching algorithms for fully automatic textual entailment recognition.…”