Large-scale Cloze Test Dataset Created by Teachers

Xie, Qizhe; Lai, Guokun; Dai, Zihang; Hovy, Eduard

doi:10.18653/v1/d18-1257

Cited by 64 publications

(49 citation statements)

References 35 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Question Answering over Documents There has been rapid progress in the task of question answering (QA) over documents along with vari- et al, 2015;Trischler et al, 2016), fictional stories (Richardson et al, 2013;Kočiskỳ et al, 2017), and textbooks (Lai et al, 2017;Xie et al, 2017). Many neural QA models have successfully addressed these tasks by leveraging coattention or bidirectional attention mechanisms (Xiong et al, 2018;Seo et al, 2017) to model the codependent context over the document and the question.…”

Section: Related Workmentioning

confidence: 99%

Efficient and Robust Question Answering from Minimal Context over Documents

Min¹,

Zhong²,

Socher³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

151

113

View full text Add to dashboard Cite

Neural models for question answering (QA) over documents have achieved significant performance improvements. Although effective, these models do not scale to large corpora due to their complex modeling of interactions between the document and the question. Moreover, recent work has shown that such models are sensitive to adversarial inputs. In this paper, we study the minimal context required to answer the question, and find that most questions in existing datasets can be answered with a small set of sentences. Inspired by this observation, we propose a simple sentence selector to select the minimal set of sentences to feed into the QA model. Our overall system achieves significant reductions in training (up to 15 times) and inference times (up to 13 times), with accuracy comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results and analyses show that our approach is more robust to adversarial inputs. * All work was done while the author was an intern at Salesforce Research.

show abstract

Section: Related Workmentioning

confidence: 99%

Efficient and Robust Question Answering from Minimal Context over Documents

Min¹,

Zhong²,

Socher³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

151

113

View full text Add to dashboard Cite

show abstract

“…In most datasets, the answer can be directly found from context. CLOTH (Xie et al, 2018) has a similar setting to ChID, where the answer should be selected from given choices. However, CLOTH is collected from English examinations for secondary/high school students, whose size is limited because documents, blanks, and options are all manually created.…”

Section: Related Workmentioning

confidence: 99%

ChID: A Large-scale Chinese IDiom Dataset for Cloze Test

Zheng¹,

Huang²,

Sun³

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Cloze-style reading comprehension in Chinese is still limited due to the lack of various corpora. In this paper we propose a large-scale Chinese cloze test dataset ChID, which studies the comprehension of idiom, a unique language phenomenon in Chinese. In this corpus, the idioms in a passage are replaced by blank symbols and the correct answer needs to be chosen from well-designed candidate idioms. We carefully study how the design of candidate idioms and the representation of idioms affect the performance of state-of-the-art models. Results show that the machine accuracy is substantially worse than that of human, indicating a large space for further research.

show abstract

“…-CLOTH Different from above automatically-generated datasets, CLOTH [93] (CLOze test by TeacHers) is human-created, which is collected from English exams for Chinese students. Questions in the CLOTH are well-designed by middle-school and high-school teachers to examine students' language proficiency including vocabulary, reasoning and grammar.…”

Section: Ms Marco[51]mentioning

confidence: 99%

“…-RACE Like CLOTH dataset [93], RACE [36] is also collected from the English exams for middle school and high school Chinese students. This corpus allows types of passages to be more various.…”

Section: -Mctestmentioning

confidence: 99%

Neural Machine Reading Comprehension: Methods and Trends

et al. 2019

View full text Add to dashboard Cite

Machine Reading Comprehension (MRC), which requires the machine to answer questions based on the given context, has gained increasingly wide attention with the incorporation of various deep learning techniques over the past few years. Although the research of MRC based on deep learning is flourishing, there remains a lack of a comprehensive survey to summarize existing approaches and recent trends, which motivates our work presented in this article. Specifically, we give a thorough review of this research field, covering different aspects including (1) typical MRC tasks: their definitions, differences and representative datasets; (2) general architecture of neural MRC: the main modules and prevalent approaches to each of them; and (3) new trends: some emerging focuses in neural MRC as well as the corresponding challenges. Last but not least, in retrospect of what has been achieved so far, the survey also envisages what the future may hold by discussing the open issues left to be addressed. * Work in progress.

show abstract

Large-scale Cloze Test Dataset Created by Teachers

Cited by 64 publications

References 35 publications

Efficient and Robust Question Answering from Minimal Context over Documents

Efficient and Robust Question Answering from Minimal Context over Documents

ChID: A Large-scale Chinese IDiom Dataset for Cloze Test

Neural Machine Reading Comprehension: Methods and Trends

Contact Info

Product

Resources

About