Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Kapanipathi, Pavan; Thost, Veronika; Patel, Siva Sankalp; Whitehead, Spencer; Abdelaziz, Ibrahim; Balakrishnan, Avinash; Chang, Maria Hsia; Fadnis, Kshitij P.; Gunasekara, R. Chulaka; Makni, Bassem; Mattei, Nicholas; Talamadupula, Kartik; Fokoue, Achille

doi:10.1609/aaai.v34i05.6318

Cited by 30 publications

(18 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, feeding the commonsense knowledge gradually during the agent's learning process provides more focus to the exploration, and drives it toward the concepts related to the rest of the goals. These results can also be seen as an RL-centric agent-based validation of similar results shown in the broader NLP literature (Kapanipathi et al 2020). We refer the reader to (Murugesan et al 2020) on further discussion on this topic.…”

Section: D2 Discussionsupporting

confidence: 77%

Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines

Murugesan¹,

Atzeni²,

Kapanipathi³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making. In this paper, we examine the problem of infusing RL agents with commonsense knowledge. Such knowledge would allow agents to efficiently act in the world by pruning out implausible actions, and to perform lookahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We also introduce several baseline RL agents which track the sequential context and dynamically retrieve the relevant commonsense knowledge from ConceptNet. We show that agents which incorporate commonsense knowledge in TWC perform better, while acting more efficiently. We conduct user-studies to estimate human performance on TWC and show that there is ample room for future improvement.

show abstract

Section: D2 Discussionsupporting

confidence: 77%

Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines

Murugesan¹,

Atzeni²,

Kapanipathi³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Traditional Attention-Based Models do not utilise contextual representations from PTLMs [7]. KG-Augmented Entailment System (KES) [11] augments the NLI model with external knowledge encoded using graph convolutional networks. Knowledge-based Inference Model (KIM) [3] incorporates lexical-level knowledge (such as synonym and antonym) into its attention and composition components.…”

Section: Related Workmentioning

confidence: 99%

“…BERT model has shown to be robust to adversarial examples when external knowledge is incorporated to the attention mechanism using simple transformations [15]. The KES [11] model highlighted above further evaluates their system with BERT contextual embeddings in the framework.…”

Section: Related Workmentioning

confidence: 99%

ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference

Gajbhiye

Moubayed

Bradley

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Neural language representation models such as BERT, pretrained on large-scale unstructured corpora lack explicit grounding to real-world commonsense knowledge and are often unable to remember facts required for reasoning and inference. Natural Language Inference (NLI) is a challenging reasoning task that relies on common human understanding of language and real-world commonsense knowledge. We introduce a new model for NLI called External Knowledge Enhanced BERT (ExBERT), to enrich the contextual representation with realworld commonsense knowledge from external knowledge sources and enhance BERT's language understanding and reasoning capabilities. ExBERT takes full advantage of contextual word representations obtained from BERT and employs them to retrieve relevant external knowledge from knowledge graphs and to encode the retrieved external knowledge. Our model adaptively incorporates the external knowledge context required for reasoning over the inputs. Extensive experiments on the challenging SciTail and SNLI benchmarks demonstrate the effectiveness of ExBERT: in comparison to the previous state-of-the-art, we obtain an accuracy of 95.9% on SciTail and 91.5% on SNLI.

show abstract

“…At the same time, these sense embeddings can also serve as an entry point to many other knowledge bases linked to WordNet, such as the multilingual knowledge graph of BabelNet (Navigli & Ponzetto, 2010), the common-sense triples of ConceptNet (Speer et al, 2017) or WebChild (Tandon et al, 2017), the semantic frames of VerbNet (Schuler, 2006), and even the images of ImageNet (Russakovsky et al, 2015) or Visual Genome (Krishna et al, 2016). Several recent works have used the symbolic relations expressed in these knowledge bases to improve neural solutions to Natural Language Inference (Kapanipathi et al, 2020), Commonsense Reasoning (Lin et al, 2019), Story Generation (Ammanabrolu et al, 2020), among others.…”

Section: Knowledge Integrationmentioning

confidence: 99%

LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond

Loureiro,

Jorge,

Camacho-Collados

2021

Preprint

View full text Add to dashboard Cite

Distributional semantics based on neural approaches is a cornerstone of Natural Language Processing, with surprising connections to human meaning representation as well. Recent Transformer-based Language Models have proven capable of producing contextual word representations that reliably convey sense-specific information, simply as a product of self-supervision. Prior work has shown that these contextual representations can be used to accurately represent large sense inventories as sense embeddings, to the extent that a distance-based solution to Word Sense Disambiguation (WSD) tasks outperforms models trained specifically for the task. Still, there remains much to understand on how to use these Neural Language Models (NLMs) to produce sense embeddings that can better harness each NLM's meaning representation abilities. In this work we introduce a more principled approach to leverage information from all layers of NLMs, informed by a probing analysis on 14 NLM variants. We also emphasize the versatility of these sense embeddings in contrast to task-specific models, applying them on several sense-related tasks, besides WSD, while demonstrating improved performance using our proposed approach over prior work focused on sense embeddings. Finally, we discuss unexpected findings regarding layer and model performance variations, and potential applications for downstream tasks.

show abstract

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Cited by 30 publications

References 19 publications

Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines

Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines

ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference

LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond

Contact Info

Product

Resources

About