KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension

Lim, Seungyoung; Kim, Myungji; Lee, Jooyoul

doi:10.48550/arxiv.1909.07005

Cited by 27 publications

(38 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DRCD [25] is a native Chinese QA dataset, consists of 30,000+ questions posed by the annotators on 10,014 paragraphs extracted from 2,108 Wikipedia articles. KorQuAD [26] is a Korean QA dataset and PIAF [27] is a French QA dataset, consisting of 70,000+ and 3835 question-answer pairs, respectively. Since large-scale QA datasets in languages other than English rarely exist and building native QA datasets is time-and cost-consuming, developing QA systems for these languages is challenging.…”

Section: B Other Languagesmentioning

confidence: 99%

“…[7] 2018 English Native 150K+ Wikiqa: A challenge dataset for open-domain question answering [8] 2015 English Native 3K+ MS MARCO: A human generated machine reading comprehension dataset [9] 2016 English Native 100K+ Natural questions: a benchmark for question answering research [10] 2019 English Native 300K+ Quac: Question answering in context [11] 2018 English Native 100K+ Coqa: A conversational question answering challenge [12] 2019 English Native 127K+ Newsqa: A machine comprehension dataset [13] 2016 English Native 100K+ Constructing datasets for multi-hop reading comprehension across documents [15] 2018 English Native, Multi-hop 50K+ Hotpotqa: A dataset for diverse, explainable multi-hop question answering [16] 2018 English Native, Multi-hop 113K+ Repartitioning of the complexwebquestions dataset [17] 2018 English Native, Multi-hop 63K+ R4C: A benchmark for evaluating RC systems to get the right answer for the right reason [18] 2019 English Native, Multi-hop 4K+ Automatic spanish translation of the squad dataset for multilingual question answering [19] 2019 Spanish Translation 100K+ Neural arabic question answering [20] 2019 Arabic Translation 48K+ Semi-supervised training data generation for multilingual question answering [21] 2018 Korean Translation 81K+ Neural learning for question answering in italian [22] 2018 Italian Translation 60K+ SberQuAD-Russian reading comprehension dataset: Description and analysis [24] 2020 Russian Native 50K+ Drcd: a chinese machine reading comprehension dataset [25] 2018 Chinese Native 30K+ Korquad1. 0: Korean qa dataset for machine reading comprehension [26] 2018 Korean Native 70K+ Project PIAF: Building a Native French Question-Answering Dataset [27] 2020 French Native 3K+ Parsinlu: a suite of language understanding challenges for persian [34] 2021 Persian Native 1K+ ParSQuAD: Persian Question Answering Dataset based on Machine Translation of SQuAD 2.0 [33] 2021 Persian Translation 25K, 70K…”

Section: B Other Languagesmentioning

confidence: 99%

See 1 more Smart Citation

PersianQuAD: The Native Question Answering Dataset for the Persian Language

2022

View full text Add to dashboard Cite

Developing Question Answering systems (QA) is one of the main goals in Artificial Intelligence. With the advent of Deep Learning (DL) techniques, QA systems have witnessed significant advances. Although DL performs very well on QA, it requires a considerable amount of annotated data for training. Many annotated datasets have been built for the QA task; most of them are exclusively in English. In order to address the need for a high-quality QA dataset in the Persian language, we present PersianQuAD, the native QA dataset for the Persian language. We create PersianQuAD in four steps: (1) Wikipedia article selection, ( 2) question-answer collection, (3) three-candidates test set preparation, and (4) Data Quality Monitoring. PersianQuAD consists of approximately 20,000 questions and answers made by native annotators on a set of Persian Wikipedia articles. The answer to each question is a segment of the corresponding article text. To better understand PersianQuAD and ensure its representativeness, we analyze PersianQuAD and show it contains questions of varying types and difficulties. We also present three versions of a deep learning-based QA system trained with PersianQuAD. Our best system achieves an F1 score of 82.97% which is comparable to that of QA systems on English SQuAD, made by the Stanford University. This shows that PersianQuAD performs well for training deep-learning-based QA systems. Human performance on PersianQuAD is significantly better (96.49%), demonstrating that PersianQuAD is challenging enough and there is still plenty of room for future improvement. PersianQuAD is freely available and can be downloaded from here. All the QA systems implemented in this paper are also available here.

show abstract

Section: B Other Languagesmentioning

confidence: 99%

Section: B Other Languagesmentioning

confidence: 99%

PersianQuAD: The Native Question Answering Dataset for the Persian Language

2022

View full text Add to dashboard Cite

show abstract

“…The schema of NSMC is similar to SST-2 (Socher et al, 2013). KorQuAD 1.0 (Lim et al, 2019) is a Korean version of machine reading comprehension dataset. 6 It consists of 10,645 training passages with 66,181 training questions and 5,774 validation questions.…”

Section: Experimental Settingmentioning

confidence: 99%

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers

Kim¹,

Kim²,

Lee³

et al. 2021

Preprint

View full text Add to dashboard Cite

GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. Here we address some remaining issues less reported by the GPT-3 paper, such as a non-English LM, the performances of different sized models, and the effect of recently introduced prompt optimization on in-context learning. To achieve this, we introduce Hy-perCLOVA, a Korean variant of 82B GPT-3 trained on a Korean-centric corpus of 560B tokens. Enhanced by our Korean-specific tokenization, HyperCLOVA with our training configuration shows state-of-the-art in-context zero-shot and few-shot learning performances on various downstream tasks in Korean. Also, we show the performance benefits of promptbased learning and demonstrate how it can be integrated into the prompt engineering pipeline. Then we discuss the possibility of materializing the No Code AI paradigm by providing AI prototyping capabilities to nonexperts of ML by introducing HyperCLOVA studio, an interactive prompt engineering interface. Lastly, we demonstrate the potential of our methods with three successful in-house applications.

show abstract

“…Another line of research uses multilingual models such as mBERT , XLM (Conneau and Lample, 2019) or XLM-Roberta (Conneau et al, 2020) trained on English QA datasets for zero-shot language transfer to the target domain. While these models perform astonishingly well on QA in unseen languages, they do not perform as well as on English QA (Lewis et al, 2020a;d'Hoffschmidt et al, 2020;Lim et al, 2019). Furthermore, multilingual models are much larger than their monolingual counterparts, rendering them unsuitable for most production systems where memory consumption and query speed matter.…”

Section: Introduction and Related Workmentioning

confidence: 99%

“…Research on non-English machine reading for question answering (QA) suffers from limited availability of annotated non-English datasets. With the English SQuAD dataset (Rajpurkar et al, 2018) as a role model, there are only a few resources of similar format, such as the French FQuAD (d'Hoffschmidt et al, 2020), the Korean KorQuAD (Lim et al, 2019), and the Russian SberQuAD (Efimov et al, 2020) datasets. As an alternative, there are machine-translated datasets for training (Lewis et al, 2020a).…”

Section: Introduction and Related Workmentioning

confidence: 99%

GermanQuAD and GermanDPR: Improving Non-English Question Answering and Passage Retrieval

Möller¹,

Risch²,

Pietsch³

2021

Preprint

View full text Add to dashboard Cite

A major challenge of research on non-English machine reading for question answering (QA) is the lack of annotated datasets. In this paper, we present GermanQuAD, a dataset of 13,722 extractive question/answer pairs. To improve the reproducibility of the dataset creation approach and foster QA research on other languages, we summarize lessons learned and evaluate reformulation of question/answer pairs as a way to speed up the annotation process. An extractive QA model trained on GermanQuAD significantly outperforms multilingual models and also shows that machine-translated training data cannot fully substitute hand-annotated training data in the target language. Finally, we demonstrate the wide range of applications of German-QuAD by adapting it to GermanDPR, a training dataset for dense passage retrieval (DPR), and train and evaluate the first non-English DPR model.

show abstract

KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension

Cited by 27 publications

References 0 publications

PersianQuAD: The Native Question Answering Dataset for the Persian Language

PersianQuAD: The Native Question Answering Dataset for the Persian Language

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers

GermanQuAD and GermanDPR: Improving Non-English Question Answering and Passage Retrieval

Contact Info

Product

Resources

About