A grapheme-level approach for constructing a Korean morphological analyzer without linguistic knowledge

Choi, Jihun; Youn, Jonghem; Lee, Sang-goo

doi:10.1109/bigdata.2016.7841061

Cited by 6 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As Korean is agglutinative (Song, 2006), the current literature in Korean word representations mainly focus on subword structures such as morphemes (Edmiston and Stratos, 2018), syllables (Choi et al, 2017) and Jamo (Choi et al, 2016;Stratos, 2017;Park et al, 2018). We here move forward one step further by incorporating Hanja information explicitly together with the aforementioned subword information.…”

Section: Related Workmentioning

confidence: 99%

Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja

Yoo

Kim

Lee

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

We propose a simple yet effective approach for improving Korean word representations using additional linguistic annotation (i.e. Hanja). We employ cross-lingual transfer learning in training word representations by leveraging the fact that Hanja is closely related to Chinese. We evaluate the intrinsic quality of representations learned through our approach using the word analogy and similarity tests. In addition, we demonstrate their effectiveness on several downstream tasks, including a novel Korean news headline generation task. * Equal contribution. 1 Hanja and traditional Chinese characters are very similar but not completely identical. Some differences in strokes and language-exclusive characters exist.

show abstract

Section: Related Workmentioning

confidence: 99%

Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja

Yoo

Kim

Lee

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

show abstract

“…Dong, et al (2016) demonstrate how radical-level features incorporated at the character-level for named entity recognition achieve state-of-the-art accuracy for Chinese. The most convincing attempt to tag Korean at the morpheme level is by Choi, et al (2016) who achieve state-of-the-art (dictionary-less) performance by using a multi-stage Bi-LSTM-CRF model that involves the splitting of Korean character input into constituent graphemes. However, the implicit assumption that Korean characters must first be split into graphemes to achieve optimal performance for morphological analysis is not well supported, and we should consider the splitting of characters into graphemes to be employing linguistic knowledge specific to Korean.…”

Section: Related Workmentioning

confidence: 99%

Rich Character-Level Information for Korean Morphological Analysis and Part-of-Speech Tagging

Matteson,

Lee,

Kim

et al. 2018

Preprint

View full text Add to dashboard Cite

Due to the fact that Korean is a highly agglutinative, character-rich language, previous work on Korean morphological analysis typically employs the use of sub-character features known as graphemes or otherwise utilizes comprehensive prior linguistic knowledge (i.e., a dictionary of known morphological transformation forms, or actions). These models have been created with the assumption that character-level, dictionary-less morphological analysis was intractable due to the number of actions required. We present, in this study, a multi-stage action-based model that can perform morphological transformation and part-of-speech tagging using arbitrary units of input and apply it to the case of character-level Korean morphological analysis. Among models that do not employ prior linguistic knowledge, we achieve state-of-the-art word and sentencelevel tagging accuracy with the Sejong Korean corpus using our proposed data-driven Bi-LSTM model.

show abstract

“…Traditional approaches to Korean morphological analysis have adopted a pipeline model of morpheme processing and POS tagging (Lee and Rim, 2009;Na, 2015;Choi et al, 2016;Matteson et al, 2018;Song and Park, 2018 Figure 1: Korean morphological analysis of a sentence "나는 하늘을 나는 새를 봤다" of which meaning is "I saw a bird flying in the sky". Correct morpheme analysis helps predicting POS tags (blue dotted arrows) while POS tagging affects morpheme analysis (red arrows).…”

Section: Introductionmentioning

confidence: 99%

Korean Morphological Analysis with Tied Sequence-to-Sequence Multi-Task Model

Song¹,

Park²

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Korean morphological analysis has been considered as a sequence of morpheme processing and POS tagging. Thus, a pipeline model of the tasks has been adopted widely by previous studies. However, the model has a problem that it cannot utilize interactions among the tasks. This paper formulates Korean morphological analysis as a combination of the tasks and presents a tied sequence-tosequence multi-task model for training the two tasks simultaneously without any explicit regularization. The experiments prove the proposed model achieves the state-of-the-art performance.

show abstract

A grapheme-level approach for constructing a Korean morphological analyzer without linguistic knowledge

Cited by 6 publications

References 10 publications

Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja

Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja

Rich Character-Level Information for Korean Morphological Analysis and Part-of-Speech Tagging

Korean Morphological Analysis with Tied Sequence-to-Sequence Multi-Task Model

Contact Info

Product

Resources

About