Commonsense Evidence Generation and Injection in Reading Comprehension

Liu, Ye; Yang, Tao; You, Zeyu; Fan, Wei; Yu, Philip S.

doi:10.48550/arxiv.2005.05240

Cited by 3 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike individual PLMs and graph encoders, KG-augmented models take both text and graph inputs. The KG-augmented model's graph encoder usually computes graph embeddings via attention pooling of nodes/paths, and the attention weights can be used to explain which nodes/paths in the input KG are salient (Lin et al, 2019;Feng et al, 2020;Liu et al, 2020;Yan et al, 2020). These KG explanations can be interpreted as identifying knowledge in the KG that is complementary to the knowledge encoded in the PLM.…”

Section: Creating Model Explanationsmentioning

confidence: 99%

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Chan

Xu²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Augmenting pre-trained language models with knowledge graphs (KGs) has achieved success on various commonsense reasoning tasks. Although some works have attempted to explain the behavior of such KG-augmented models by indicating which KG inputs are salient (i.e., important for the model's prediction), it is not always clear how these explanations should be used to make the model better. In this paper, we explore whether KG explanations can be used as supervision for teaching these KG-augmented models how to filter out unhelpful KG information. To this end, we propose SALKG, a simple framework for learning from KG explanations of both coarse (Is the KG salient?) and fine (Which parts of the KG are salient?) granularity. Given the explanations generated from a task's training set, SALKG trains KG-augmented models to solve the task by focusing on KG information highlighted by the explanations as salient. Across two popular commonsense QA benchmarks and three KG-augmented models, we find that SALKG's training process can consistently improve model performance. 1 * Equal contribution. † Work done while TG interned remotely at USC. 1 Code will be released at github.com/INK-USC/SalKG.

show abstract

Section: Creating Model Explanationsmentioning

confidence: 99%

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Chan

Xu²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots

Zhao¹,

Liu²,

Wan³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Few-shot table-to-text generation is a task of composing fluent and faithful sentences to convey table content using limited data. Despite many efforts having been made towards generating impressive fluent sentences by finetuning powerful pre-trained language models, the faithfulness of generated content still needs to be improved. To this end, this paper proposes a novel approach Attend, Memorize and Generate (called AMG), inspired by the text generation process of humans. In particular, AMG (1) attends over the multi-granularity of context using a novel strategy based on table slot level and traditional token-by-token level attention to exploit both the table structure and natural linguistic information; (2) dynamically memorizes the table slot allocation states; and (3) generates faithful sentences according to both the context and memory allocation states. Comprehensive experiments with human evaluation on three domains (i.e., humans, songs, and books) of the Wiki dataset show that our model can generate higher qualified texts when compared with several state-ofthe-art baselines, in both fluency and faithfulness. 1

show abstract

KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning

Liu¹,

Wan²,

He³

et al. 2020

Preprint

View full text Add to dashboard Cite

Generative commonsense reasoning which aims to empower machines to generate sentences with the capacity of reasoning over a set of concepts is a critical bottleneck for text generation. Even the state-of-the-art pre-trained language generation models struggle at this task and often produce implausible and anomalous sentences. One reason is that they rarely consider incorporating the knowledge graph which can provide rich relational information among the commonsense concepts. To promote the ability of commonsense reasoning for text generation, we propose a novel knowledge graphaugmented pre-trained language generation model KG-BART, which encompasses the complex relations of concepts through the knowledge graph and produces more logical and natural sentences as output. Moreover, KG-BART can leverage the graph attention to aggregate the rich concept semantics that enhances the model generalization on unseen concept sets. Experiments on benchmark CommonGen dataset verify the effectiveness of our proposed approach by comparing with several strong pre-trained language generation models, particularly KG-BART outperforms BART by 15.98%, 17.49%, in terms of BLEU-3, 4. Moreover, we also show that the generated context by our model can work as background scenarios to benefit downstream commonsense QA tasks.

show abstract

Commonsense Evidence Generation and Injection in Reading Comprehension

Cited by 3 publications

References 18 publications

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots

KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning

Contact Info

Product

Resources

About