An Overview of Utilizing Knowledge Bases in Neural Networks for Question Answering

Kafle, Sabin; Silva, Nisansa de; Dou, Dejing

doi:10.1109/iri.2019.00058

Cited by 4 publications

(2 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During training, we infuse the model with knowledge from an external source, distinct from datasets of paired V&L data.The challenge is that standard V&L data is not annotated or paired with such additional knowledge. Even though techniques have been proposed to exploit additional data in NLP, including text-based question answering (Kafle et al, 2019;Lv et al, 2019;Lin et al, 2019;Rajani et al, 2019), little work has been done on the extension to V&L. Works in NLP with benchmarks of knowledge-demanding questions (Yang et al, 2015;Talmor et al, 2018;Mihaylov et al, 2018;Sap et al, 2019) have shown that knowledge bases (KBs) contain information that can benefit large-scale pretrained models. Motivated by this line of evidence, we aim to evaluate similar mechanisms for V&L tasks.…”

Section: Introductionmentioning

confidence: 99%

Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge

Shevchenko¹,

Teney²,

Dick³

et al. 2021

Preprint

View full text Add to dashboard Cite

The limits of applicability of vision-andlanguage models are defined by the coverage of their training data. Tasks like vision question answering (VQA) often require commonsense and factual information beyond what can be learned from task-specific datasets. This paper investigates the injection of knowledge from general-purpose knowledge bases (KBs) into vision-and-language transformers. We use an auxiliary training objective that encourages the learned representations to align with graph embeddings of matching entities in a KB. We empirically study the relevance of various KBs to multiple tasks and benchmarks. The technique brings clear benefits to knowledge-demanding question answering tasks (OK-VQA, FVQA) by capturing semantic and relational knowledge absent from existing models. More surprisingly, the technique also benefits visual reasoning tasks (NLVR2, SNLI-VE). We perform probing experiments and show that the injection of additional knowledge regularizes the space of embeddings, which improves the representation of lexical and semantic similarities. The technique is model-agnostic and can expand the applicability of any vision-and-language transformer with minimal computational overhead.

show abstract

Section: Introductionmentioning

confidence: 99%

Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge

Shevchenko¹,

Teney²,

Dick³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…can be answered by the fact "(Estella Warren, people.person.nationality, Canada)," where the tail entity "Canada" is returned as the answer. As demonstrated by Kafle et al (2020), KBQA has a great practical application potential.…”

Section: Introductionmentioning

confidence: 99%

Robust cross-lingual knowledge base question answering via knowledge distillation

Shaofei

Dang

2021

DTA

View full text Add to dashboard Cite

PurposePrevious knowledge base question answering (KBQA) models only consider the monolingual scenario and cannot be directly extended to the cross-lingual scenario, in which the language of questions and that of knowledge base (KB) are different. Although a machine translation (MT) model can bridge the gap through translating questions to the language of KB, the noises of translated questions could accumulate and further sharply impair the final performance. Therefore, the authors propose a method to improve the robustness of KBQA models in the cross-lingual scenario.Design/methodology/approachThe authors propose a knowledge distillation-based robustness enhancement (KDRE) method. Specifically, first a monolingual model (teacher) is trained by ground truth (GT) data. Then to imitate the practical noises, a noise-generating model is designed to inject two types of noise into questions: general noise and translation-aware noise. Finally, the noisy questions are input into the student model. Meanwhile, the student model is jointly trained by GT data and distilled data, which are derived from the teacher when feeding GT questions.FindingsThe experimental results demonstrate that KDRE can improve the performance of models in the cross-lingual scenario. The performance of each module in KBQA model is improved by KDRE. The knowledge distillation (KD) and noise-generating model in the method can complementarily boost the robustness of models.Originality/valueThe authors first extend KBQA models from monolingual to cross-lingual scenario. Also, the authors first implement KD for KBQA to develop robust cross-lingual models.

show abstract

Perspectives and Views of Flexible Query Answering

Andreasen¹,

Tré

Kacprzyk

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

An Overview of Utilizing Knowledge Bases in Neural Networks for Question Answering

Cited by 4 publications

References 52 publications

Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge

Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge

Robust cross-lingual knowledge base question answering via knowledge distillation

Perspectives and Views of Flexible Query Answering

Contact Info

Product

Resources

About