Open Korean Corpora: A Practical Report

Abstract

Korean is often referred to as a low-resource language in the research community. While this claim is partially true, it is also because the availability of resources is inadequately advertised and curated. This work curates and reviews a list of Korean corpora, first describing institution-level resource development, then further iterate through a list of current open datasets for different types of tasks. We then propose a direction on how open-source dataset construction and releases should be done for less-resourced languages to promote research.

A Survey on Awesome Korean NLP Datasets

“…Won Ik Cho, et al included ClovaCall dataset in their work on Open Korean Corpora [40]. Jihwan Bang, et al followed a variant version of LAS model proposed on ClovaCall paper [50].…”

“…KMOUNLP-NER dataset was processed with the tokenization method on Korean sentences from various resources such as news articles. [40]. So-Yeop Yoo and Jeong Ok-Ran applied BERT and knowledge graph method for Korean contextual information extraction, trained with KMOUNLP-NER for named entity recognition in 2020 [43].…”

“…Although the size of this dataset is small, the reliability of this dataset is quite high because the summarized sentences are written by human, without any augmentation algorithm. This dataset was enlisted on Open Korean Corpora [40]. KLUE consists of 8 different large datasets for various NLP tasks.…”

“…ParaKQC contributed as a resource of KLUE dataset [24]. Also it is enlisted on Open Korean Corpora [40]. Won Ik Cho, et al applied the sentence classification scheme of ParaKQC for StyleKQC, a stylevariant paraphrase corpus for Korean question and commands [67].…”

“…Cho, Won Ik, et al established Open Korean Corpora [40] to provide assorted Korean NLP datasets. They provide the summarized detail of 29 different datasets from text parsing and tagging area, sentence similarity, sentence classification and QA, parallel corpora, multilingual corpora, and speech corpora.…”