Text Understanding with the Attention Sum Reader Network

Kadlec, Rudolf; Schmid, Martin; Bajgar, Ondřej; Kleindienst, Jan

doi:10.18653/v1/p16-1086

Cited by 247 publications

(255 citation statements)

References 10 publications

Supporting

Mentioning

255

Contrasting

Order By: Relevance

“…Hill et al (2016) propose a windowbased memory network for CBT dataset. Kadlec et al (2016) introduce pointer networks with one attention step to predict the blanking out entities. Sordoni et al (2016) propose an iterative alternating attention mechanism to better model the links between question and passage.…”

Section: Related Workmentioning

confidence: 99%

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Wang

Yang

Wei

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

622

420

View full text Add to dashboard Cite

In this paper, we present the gated selfmatching networks for reading comprehension style question answering, which aims to answer questions from a given passage. We first match the question and passage with gated attention-based recurrent networks to obtain the question-aware passage representation. Then we propose a self-matching attention mechanism to refine the representation by matching the passage against itself, which effectively encodes information from the whole passage. We finally employ the pointer networks to locate the positions of answers from the passages. We conduct extensive experiments on the SQuAD dataset. The single model achieves 71.3% on the evaluation metrics of exact match on the hidden test set, while the ensemble model further boosts the results to 75.9%. At the time of submission of the paper, our model holds the first place on the SQuAD leaderboard for both single and ensemble model.

show abstract

Section: Related Workmentioning

confidence: 99%

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Wang

Yang

Wei

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

622

420

View full text Add to dashboard Cite

show abstract

“…Our Chengyu cloze test task is similar to reading comprehension (Hermann et al, 2015;Cui et al, 2016;Kadlec et al, 2016;Seo et al, 2016 (Xu et al, 2010) and improve Chinese word segmentation (Chan and Chong, 2008;Sun and Xu, 2011;Wang and Xu, 2017). Chengyus differ from metaphors in other languages (Tsvetkov et al, 2014;Shutova, 2010) because they do not follow the grammatical structure and syntax of the modern Chinese.…”

Section: Related Workmentioning

confidence: 99%

Chengyu Cloze Test

Jiang¹,

Zhang²,

Huang³

et al. 2018

Proceedings of the Thirteenth Workshop on Innovative Use of NLP For Building Educational Applications

View full text Add to dashboard Cite

We present a neural recommendation model for Chengyu, which is a special type of Chinese idiom. Given a query, which is a sentence with an empty slot where the Chengyu is taken out, our model will recommend the Chengyu candidate that best fits the slot context. The main challenge lies in that the literal meaning of a Chengyu is usually very different from its figurative meaning. We propose a neural approach to incorporate the definition of each Chengyu as background knowledge. Experiments on both Chengyu cloze test and coherence checking in college entrance exams show that our system achieves 89.5% accuracy on cloze test and outperforms human experts who attended competitive universities in China. We will make all of our data sets and resources publicly available as a new benchmark for research purposes 1 .

show abstract

“…These tasks include but are not limited to: performing reasoning over a simulated environment for QA , factoid and non-factoid based QA using both knowledge bases and unstructured text (Kumar et al, 2015;Hill et al, 2016;Chandar et al, 2016;Bordes et al, 2015), goal driven dialog Dodge et al, 2016;Weston, 2016), automatic story comprehension from both video and text (Tapaswi et al, 2015), and, transferring knowledge from one knowledge-base while learning to answer questions on a different knowledge base (Bordes et al, 2015). Recently, various other attention based neural models (similar to Memory Networks) have been proposed to tackle the machine comprehension task by QA from unstructured text (Kadlec et al, 2016;Sordoni et al, 2016;Chen et al, 2016). To the best of our knowledge, knowledge transfer from an unstructured text dataset to another unstructured text dataset for machine comprehension is not explored yet.…”

Section: Related Workmentioning

confidence: 99%

Learning and Knowledge Transfer with Memory Networks for Machine Comprehension

Yadav¹,

Vig

Shroff

2017

Proceedings of the 15th Conference of the European Chapter of The Association for Computational Linguistics: Volume 1

View full text Add to dashboard Cite

Enabling machines to read and comprehend unstructured text remains an unfulfilled goal for NLP research. Recent research efforts on the "machine comprehension" task have managed to achieve close to ideal performance on simulated data. However, achieving similar levels of performance on small real world datasets has proved difficult; major challenges stem from the large vocabulary size, complex grammar, and the frequent ambiguities in linguistic structure. On the other hand, the requirement of human generated annotations for training, in order to ensure a sufficiently diverse set of questions is prohibitively expensive. Motivated by these practical issues, we propose a novel curriculum inspired training procedure for Memory Networks to improve the performance for machine comprehension with relatively small volumes of training data. Additionally, we explore various training regimes for Memory Networks to allow knowledge transfer from a closely related domain having larger volumes of labelled data. We also suggest the use of a loss function to incorporate the asymmetric nature of knowledge transfer. Our experiments demonstrate improvements on Dailymail, CNN, and MCTest datasets.

show abstract

Text Understanding with the Attention Sum Reader Network

Cited by 247 publications

References 10 publications

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Chengyu Cloze Test

Learning and Knowledge Transfer with Memory Networks for Machine Comprehension

Contact Info

Product

Resources

About