Entailment and anaphora resolution in RTE3

Abstract

We present VENSES, a linguistically-based approach for semantic inference which is built around a neat division of labour between two main components: a grammatically-driven subsystem which is responsible for the level of predicatearguments well-formedness and works on the output of a deep parser that produces augmented head-dependency structures. A second subsystem fires allowed logical and lexical inferences on the basis of different types of structural transformations intended to produce a semantically valid meaning correspondence. In the current challenge, we produced a new version of the system, where we do away with grammatical relations and only use semantic roles to generate weighted scores. We also added a number of additional modules to cope with fine-grained inferential triggers which were not present in previous dataset. Different levels of argumenthood have been devised in order to cope with semantic uncertainty generated by nearly-inferrable Text-Hypothesis pairs where the interpretation needs reasoning. RTE3 has introduced texts of paragraph length: in turn this has prompted us to upgrade VENSES by the addition of a discourse level anaphora resolution module, which is paramount to allow entailment in pairs where the relevant portion of text contains pronominal expressions. We present the system, its relevance to the task at hand and an evaluation.

Language Varieties of Italy: Technology Challenges and Opportunities

“…Natural language processing research for specific languages and dialects of Italy is scarce and scattered across disciplines. The most studied language variety is Venetian, for which there exist work on morphological analysis (Tonelli et al, 2010), part-of-speech tagging (POS; Jaber et al, 2011), word sense disambiguation (Conforti and Fraser, 2017), and a preliminary investigation on Venetian-English machine translation (MT; Delmonte et al, 2009). Ligurian has also recently gained attention in NLP, with work on text normalization (Lusito et al, 2023) and the development of a Universal Dependency (UD; de Marneffe et al, 2021) treebank for the Genoese variety (Lusito and Maillard, 2021).…”

Arnli: Arabic Natural Language Inference Entailment and Contradiction Detection

“…Each pair was labeled by human annotators as a paraphrase or not. [14,35] Wang et al [49] proposed GLUE 13 (general language understanding evaluation benchmark) -a tool for evaluating and analyzing the performance of models across a diverse range of existing NLU tasks based on NLI. Moreover, Wang et al [48] proposed SuperGlue [46] as an improvement on Glue by having more challenging tasks, more diverse task formats, and so on.…”

“…The main impact is that RTE can transfer a problem from text data set language processing to algebra sets and logical implications; for this reason, RTE has a significant influence when added as a component in many NLP applications, as it can simplify problems. Textual inference is a key capability for improving the performance of a wide range of NLP tasks [43], such as question-answering systems [42], Information Retrieval (IR) and Information Extraction (IE) 1 , text summarization, 2 next-generation information retrieval [42], machine reading [12,36], machine translation [37], Natural Language Understanding (NLU) [51], anaphora resolution [13], and argumentation mining [29]. Since 2005, several challenges have been coordinated with the aim to provide concrete data sets that the research community could use to test and compare their different approaches to recognizing entailments.…”

Recognizing Textual Entailment with a Semantic Edit Distance Metric

“…The work was partially supported by the Mexican Government, specifically, by the Mexican National Council for Science and Technology (CONACYT) via scholarship reference 309261 to the first author and CONACYT 50206-H and SIP-IPN 20121823 projects; by Governments of India and Mexico under the CONACYT-DST India (CONACYT Table V COMPARISON WITH PREVIOUS WORKS OVER THE RTE TEST DATASETS Method RTE-1 (Accuracy) RTE-2 (Accuracy) RTE-3 (Accuracy) Roth and Sammons [4] --65.56% Burchardt and Frank [12], Burchardt et al [5] 54.6% 59.8% 62.62% Delmonte et al [6], Delmonte et al [7], Delmonte et al [13] 59 …”