A Strong Lexical Matching Method for the Machine Comprehension Test

Smith, Ellery; Greco, N.; Bošnjak, Matko; Vlachos, Andreas

doi:10.18653/v1/d15-1197

Cited by 26 publications

(28 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although this skill can be regarded as part of elaboration, we defined it as an independent skill because this knowledge is specific to RC. We were motivated by the discussion in Smith et al (2015).…”

Section: Prerequisite Skillsmentioning

confidence: 99%

Evaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability

Sugawara¹,

Kido²,

Yokono³

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Knowing the quality of reading comprehension (RC) datasets is important for the development of natural-language understanding systems. In this study, two classes of metrics were adopted for evaluating RC datasets: prerequisite skills and readability. We applied these classes to six existing datasets, including MCTest and SQuAD, and highlighted the characteristics of the datasets according to each metric and the correlation between the two classes. Our dataset analysis suggests that the readability of RC datasets does not directly affect the question difficulty and that it is possible to create an RC dataset that is easy to read but difficult to answer.

show abstract

Section: Prerequisite Skillsmentioning

confidence: 99%

Evaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability

Sugawara¹,

Kido²,

Yokono³

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

show abstract

“…On very small datasets such as MCTest-160 and MCTest-500, it is not feasible to train memory network (Smith et al, 2015), therefore, we explore the use of word vectors from the embedding matrix of a model pre-trained on CNN datasets. Here, the embedding matrix refers to the encoding matrix A used in the first step of memory network as mentioned in Section 4.…”

Section: Resultsmentioning

confidence: 99%

Learning and Knowledge Transfer with Memory Networks for Machine Comprehension

Yadav¹,

Vig

Shroff

2017

Proceedings of the 15th Conference of the European Chapter of The Association for Computational Linguistics: Volume 1

View full text Add to dashboard Cite

Enabling machines to read and comprehend unstructured text remains an unfulfilled goal for NLP research. Recent research efforts on the "machine comprehension" task have managed to achieve close to ideal performance on simulated data. However, achieving similar levels of performance on small real world datasets has proved difficult; major challenges stem from the large vocabulary size, complex grammar, and the frequent ambiguities in linguistic structure. On the other hand, the requirement of human generated annotations for training, in order to ensure a sufficiently diverse set of questions is prohibitively expensive. Motivated by these practical issues, we propose a novel curriculum inspired training procedure for Memory Networks to improve the performance for machine comprehension with relatively small volumes of training data. Additionally, we explore various training regimes for Memory Networks to allow knowledge transfer from a closely related domain having larger volumes of labelled data. We also suggest the use of a loss function to incorporate the asymmetric nature of knowledge transfer. Our experiments demonstrate improvements on Dailymail, CNN, and MCTest datasets.

show abstract

“…We also tried the improved SW and WD algorithms proposed in (Smith et al, 2015), and the system performance has improvement. Sliding-window and Word Distance-based algorithms are are described as follows:…”

Section: Two Rule-based Baselinesmentioning

confidence: 99%

ECNU at SemEval-2018 Task 11: Using Deep Learning Method to Address Machine Comprehension Task

Sheng¹,

Lan

Wu³

2018

Proceedings of the 12th International Workshop on Semantic Evaluation

View full text Add to dashboard Cite

This paper describes the system we submitted to the Task 11 in SemEval 2018, i.e., Machine Comprehension using Commonsense Knowledge. Given a passage and some questions that each have two candidate answers, this task requires the participate system to select out one answer meet the meaning of original text or commonsense knowledge from the candidate answers. For this task, we use a deep learning method to obtain final predict answer by calculating relevance of choices representations and question-aware document representation.

show abstract

A Strong Lexical Matching Method for the Machine Comprehension Test

Cited by 26 publications

References 13 publications

Evaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability

Evaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability

Learning and Knowledge Transfer with Memory Networks for Machine Comprehension

ECNU at SemEval-2018 Task 11: Using Deep Learning Method to Address Machine Comprehension Task

Contact Info

Product

Resources

About