Towards Inference-Oriented Reading Comprehension: ParallelQA

Wadhwa, Soumya; Embar, Varsha; Grabmair, Matthias; Nyberg, Eric

doi:10.18653/v1/w18-1001

Cited by 7 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rodrigues et al (2018) found declarative CQA sentence pairs to be more difficult to resolve than interrogative pairs as the latter contain more cases of superficial overlap. In addition, Wadhwa et al (2018b) showed that competitive neural reading comprehension models are susceptible to shallow patterns (e.g. lexical overlap).…”

Section: Introductionmentioning

confidence: 99%

Aiming beyond the Obvious: Identifying Non-Obvious Cases in Semantic Similarity Datasets

Peinelt¹,

Liakata

Nguyen

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Existing datasets for scoring text pairs in terms of semantic similarity contain instances whose resolution differs according to the degree of difficulty. This paper proposes to distinguish obvious from non-obvious text pairs based on superficial lexical overlap and ground-truth labels. We characterise existing datasets in terms of containing difficult cases and find that recently proposed models struggle to capture the non-obvious cases of semantic similarity. We describe metrics that emphasise cases of similarity which require more complex inference and propose that these are used for evaluating systems for semantic similarity.

show abstract

Section: Introductionmentioning

confidence: 99%

Aiming beyond the Obvious: Identifying Non-Obvious Cases in Semantic Similarity Datasets

Peinelt¹,

Liakata

Nguyen

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…The paper [3] discusses a new method of creating a Reading Comprehension system that is not limited to basic lexical pattern matching between passages and answers. The main goal of the paper is to create a Reading Comprehension System that uses reference resolution, multiple steps of reasoning, and also uses world knowledge.…”

Section: Reading Comprehensionmentioning

confidence: 99%

“…To prevent the model from suffering from annotation bias, the questions and answers were framed by different people. Similar to the collection of data in [1] and [3], the Kannada extracts were distributed to crowd source workers and they framed the questions and annotated the answers.…”

Section: Data Collectionmentioning

confidence: 99%

Development of Reading Comprehension System for Kannada Text Documents

2019

IJITEE

View full text Add to dashboard Cite

Reading Comprehension (RC) plays an important role in Natural Language Processing (NLP) as it reads and understands text written in Natural Language. Reading Comprehension systems comprehend the given document and answer questions in the context of the given document. This paper proposes a Reading Comprehension System for Kannada documents. The RC system analyses text in the Kannada script and allows users to pose questions to It in Kannada. This system is aimed at masses whose primary language is Kannada - who would otherwise have difficulties in parsing through vast Kannada documents for the information they require. This paper discusses the proposed model built using Term Frequency - Inverse Document Frequency (TF-IDF) and its performance in extracting the answers from the context document. The proposed model captures the grammatical structure of Kannada to provide the most accurate answers to the user

show abstract

“…These insights can be useful for developing benchmarks and datasets which enable realistic evaluation of systems which aim to 'solve' the RC task. In Wadhwa et al (2018), we take a first step in this direction by proposing a method focused on questions involving referential inference, a setting to which these models fail to generalize well.…”

Section: What Conditions Must Be Metmentioning

confidence: 99%

Comparative Analysis of Neural QA models on SQuAD

Wadhwa¹,

Chandu

Nyberg

2018

Proceedings of the Workshop on Machine Reading for Question Answering

Self Cite

View full text Add to dashboard Cite

The task of Question Answering has gained prominence in the past few decades for testing the ability of machines to understand natural language. Large datasets for Machine Reading have led to the development of neural models that cater to deeper language understanding compared to information retrieval tasks. Different components in these neural architectures are intended to tackle different challenges. As a first step towards achieving generalization across multiple domains, we attempt to understand and compare the peculiarities of existing end-to-end neural models on the Stanford Question Answering Dataset (SQuAD) by performing quantitative as well as qualitative analysis of the results attained by each of them. We observed that prediction errors reflect certain model-specific biases, which we further discuss in this paper.

show abstract

Towards Inference-Oriented Reading Comprehension: ParallelQA

Cited by 7 publications

References 27 publications

Aiming beyond the Obvious: Identifying Non-Obvious Cases in Semantic Similarity Datasets

Aiming beyond the Obvious: Identifying Non-Obvious Cases in Semantic Similarity Datasets

Development of Reading Comprehension System for Kannada Text Documents

Comparative Analysis of Neural QA models on SQuAD

Contact Info

Product

Resources

About