Dynamically Fused Graph Network for Multi-hop Reasoning

Xiao, Yunxuan; Qu, Yanru; Qiu, Lin; Zhou, Hao; Li, Lei; Zhang, Weinan; Yu, Yong

doi:10.48550/arxiv.1905.06933

Cited by 17 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Guessing has been found unduly successful for the widely used SQuAD QA dataset [9,34]. The complex QA benchmark HotPotQA [48] was supposed to test multi-step reasoning. However, Min et al [28] found that top-scoring QA systems as well as humans could guess the answer without such reasoning for a large fraction of queries.…”

Section: Introductionmentioning

confidence: 99%

Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answering

Jain,

Kothyari,

Kumar

et al. 2021

Preprint

View full text Add to dashboard Cite

Multimodal IR, spanning text corpus, knowledge graph and images, called outside knowledge visual question answering (OKVQA), is of much recent interest. However, the popular data set has serious limitations. A surprisingly large fraction of queries do not assess the ability to integrate cross-modal information. Instead, some are independent of the image, some depend on speculation, some require OCR or are otherwise answerable from the image alone. To add to the above limitations, frequency-based guessing is very effective because of (unintended) widespread answer overlaps between train and test folds. Overall, it is hard to determine when state-of-theart systems exploit these weaknesses rather than really infer the answers, because they are opaque and their 'reasoning' process is uninterpretable. An equally important limitation is that the dataset is designed for the quantitative assessment only of the end-to-end answer retrieval task, with no provision for assessing the correct (semantic) interpretation of the input query. In response, we identify a key structural idiom in OKVQA, viz., S3 (select, substitute and search), and build a new data set and challenge around it. Specifically, the questioner identifies an entity in the image and asks a question involving that entity which can be answered only by consulting a knowledge graph or corpus passage mentioning the entity. Our challenge consists of (i)OKVQA S3 , a subset of OKVQA annotated based on the structural idiom and (ii)S3VQA, a new dataset built from scratch. We also present a neural but structurally transparent OKVQA system, S3, that explicitly addresses our challenge data set, and outperforms recent competitive baselines. We make our code and data available at https://s3vqa.github.io/ * Both authors contributed equally to this research.

show abstract

Section: Introductionmentioning

confidence: 99%

Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answering

Jain,

Kothyari,

Kumar

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The graph connects documents that mention the same named entities. To improve scalability, the Dynamically Fused Graph Network (DFGN) (Xiao et al, 2019) adopts a dynamic construction of the graph, starting from the entities in the question and gradually selecting the supporting facts. Similarly, Ding et al (2019) implement a dynamic graph exploration inspired by the dual-process theory (Evans, 2003;Sloman, 1996;Evans, 1984).…”

Section: Hybrid Modelsmentioning

confidence: 99%

A Survey on Explainability in Machine Reading Comprehension

Thayaparan,

Valentino,

Freitas

2020

Preprint

View full text Add to dashboard Cite

This paper presents a systematic review of benchmarks and approaches for explainability in Machine Reading Comprehension (MRC). We present how the representation and inference challenges evolved and the steps which were taken to tackle these challenges. We also present the evaluation methodologies to assess the performance of explainable systems. In addition, we identify persisting open research questions and highlight critical directions for future work.

show abstract

“…Answer-aware GCN First we calculate an answer-aware sub-graph, where irrelevant entities are masked out, only those entity nodes related to answer are allowed to disseminate information. Similar to Xiao et al (2019), a soft mask…”

Section: Entity Graph Constructionmentioning

confidence: 99%

“…In addition to the cross-entropy loss, we also introduce Breadth-First Search (BFS) Loss (Xiao et al, 2019) which is a weakly supervised loss to further assist the training procedure. Given the answer entities, we conduct the BFS over the adjacent matrices of the entity graph we build to obtain heuristic masks as a weak supervision signal.…”

Section: Breadth-first Search Lossmentioning

confidence: 99%

“…Multi-hop QA Popular Graph Nueral Network (GNN) frameworks, such as graph convolutional networks (Kipf and Welling, 2016), graph attention network (Veličković et al, 2017), and graph recurrent network (Song et al, 2018) have been explored and showed promising results on multi-hop QA task that requiring reasoning. Xiao et al (2019) proposes a dynamic fused graph network to work on multi-hop QA on the HotpotQA dataset. De Cao et al ( 2018) proposes an entity-GCN method to reason over across multiple documents for multi-hop QA on the WIKIHOP dataset (Welbl et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multi-hop Question Generation with Graph Convolutional Network

Su,

Xu,

Dai

et al. 2020

Preprint

View full text Add to dashboard Cite

Multi-hop Question Generation (QG) aims to generate answer-related questions by aggregating and reasoning over multiple scattered evidence from different paragraphs. It is a more challenging yet under-explored task compared to conventional single-hop QG, where the questions are generated from the sentence containing the answer or nearby sentences in the same paragraph without complex reasoning. To address the additional challenges in multi-hop QG, we propose Multi-Hop Encoding Fusion Network for Question Generation (MulQG), which does context encoding in multiple hops with Graph Convolutional Network and encoding fusion via an Encoder Reasoning Gate. To the best of our knowledge, we are the first to tackle the challenge of multi-hop reasoning over paragraphs without any sentence-level information. Empirical results on HotpotQA dataset demonstrate the effectiveness of our method, in comparison with baselines on automatic evaluation metrics. Moreover, from the human evaluation, our proposed model is able to generate fluent questions with high completeness and outperforms the strongest baseline by 20.8% in the multihop evaluation. The code is publicly available at https://github.com/HLTCHKUST/MulQG.

show abstract

Dynamically Fused Graph Network for Multi-hop Reasoning

Cited by 17 publications

References 20 publications

Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answering

Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answering

A Survey on Explainability in Machine Reading Comprehension

Multi-hop Question Generation with Graph Convolutional Network

Contact Info

Product

Resources

About