Ensemble Approach for Natural Language Question Answering Problem

Aniol, Anna; Pietroń, Marcin; Duda, Jerzy

doi:10.1109/candarw.2019.00039

Cited by 15 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…What questions account for the largest proportion of 49.97%. Compared to SQuAD, the percentage of the What question in our dataset is similar to that in SQuAD (53.60%) (Aniol et al, 2019).…”

Section: Type-based Analysismentioning

confidence: 64%

A Vietnamese Dataset for Evaluating Machine Reading Comprehension

Nguyen

Nguyen³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

138

View full text Add to dashboard Cite

Over 97 million people speak Vietnamese as their native language in the world. However, there are few research studies on machine reading comprehension (MRC) for Vietnamese, the task of understanding a text and answering questions related to it. Due to the lack of benchmark datasets for Vietnamese, we present the Vietnamese Question Answering Dataset (UIT-ViQuAD), a new dataset for the low-resource language as Vietnamese to evaluate MRC models. This dataset comprises over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia. In particular, we propose a new process of dataset creation for Vietnamese MRC. Our in-depth analyses illustrate that our dataset requires abilities beyond simple reasoning like word matching and demands single-sentence and multiple-sentence inferences. Besides, we conduct experiments on state-of-the-art MRC methods for English and Chinese as the first experimental models on UIT-ViQuAD. We also estimate human performance on the dataset and compare it to the experimental results of powerful machine learning models. As a result, the substantial differences between human performance and the best model performance on the dataset indicate that improvements can be made on UIT-ViQuAD in future research. Our dataset is freely available on our website 1 to encourage the research community to overcome challenges in Vietnamese MRC.

show abstract

Section: Type-based Analysismentioning

confidence: 64%

A Vietnamese Dataset for Evaluating Machine Reading Comprehension

Nguyen

Nguyen³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

138

View full text Add to dashboard Cite

show abstract

“…The table show that the question type What accounted for the largest proportion with 53.09%. Compared to the SQuAD dataset, the rate of the What question in our dataset is similar to that in SQuAD (53.60%) (Aniol et al, 2019). Our corpus requires abilities beyond factoid questions that demand intricate knowledge and skills to answer like Why and How questions.…”

Section: Article Lengthmentioning

confidence: 67%

New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles

Nguyen¹,

Huynh²,

Nguyen³

et al. 2020

Preprint

View full text Add to dashboard Cite

Although over 95 million people in the world speak the Vietnamese language, there are not any large and qualified datasets for automatic reading comprehension. In addition, machine reading comprehension for the health domain offers great potential for practical applications; however, there is still very little machine reading comprehension research in this domain. In this study, we present ViNewsQA as a new corpus for the low-resource Vietnamese language to evaluate models of machine reading comprehension. The corpus comprises 10,138 human-generated question-answer pairs. Crowdworkers created the questions and answers based on a set of over 2,030 online Vietnamese news articles from the VnExpress news website, where the answers comprised spans extracted from the corresponding articles. In particular, we developed a process of creating a corpus for the Vietnamese language. Comprehensive evaluations demonstrated that our corpus requires abilities beyond simple reasoning such as word matching, as well as demanding difficult reasoning similar to inferences based on single-or-multiple-sentence information. We conducted experiments using state-of-the-art methods for machine reading comprehension to obtain the first baseline performance measures, which will be compared with further models' performances. We measured human performance based on the corpus and compared it with several strong neural models. Our experiments showed that the best model was BERT, which

show abstract

“…Ensembles in NLP Ensembles applied to NLP problems are proven to be effective techniques to improve the quality of the final model, for example, in [Aniol et al, 2019]. One of the main reasons for that is the large amount of data required for each NLP problem, that is presented in [Liu et al, 2019], [Chikhi et al, 2023], and [Wang et al, 2022] and the process requires effectively combining the data for final inference.…”

Section: Related Workmentioning

confidence: 99%

Effective Training-Time Stacking for Ensembling of Deep Neural Networks

Proskura

Zaytsev

2022

Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition

View full text Add to dashboard Cite

Ensembles are important tools for improving the performance of machine learning models. In cases related to natural language processing, ensembles boost the performance of a method due to multiple large models available in open source. However, existing approaches mostly rely on simple averaging of predictions by ensembles with equal weights for each model, ignoring differences in the quality and conformity of models. We propose to estimate weights for ensembles of NLP models using not only knowledge of their individual performance but also their similarity to each other. By adopting distance measures based on Topological Data Analysis (TDA), we improve our ensemble. The quality improves for both text classification accuracy and relevant uncertainty estimation.

show abstract

Ensemble Approach for Natural Language Question Answering Problem

Cited by 15 publications

References 10 publications

A Vietnamese Dataset for Evaluating Machine Reading Comprehension

A Vietnamese Dataset for Evaluating Machine Reading Comprehension

New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles

Effective Training-Time Stacking for Ensembling of Deep Neural Networks

Contact Info

Product

Resources

About