KR-WordRank : An Unsupervised Korean Word Extraction Method Based on WordRank

Kim, Hyun‐Joong; Cho, Sungzoon; Kang, Pilsung

doi:10.7232/jkiie.2014.40.1.018

Cited by 6 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods of using affixes to determine Korean word boundaries can be found in existing studies [12], [21]. The study in [21] is similar in that it proposes a method which attempts to exclude meaningless morphemes from the analysis results, but it does not use the number of types of affixes which can be combined with roots.…”

Section: B Methods For Finding Word Boundariesmentioning

confidence: 99%

“…The study in [21] is similar in that it proposes a method which attempts to exclude meaningless morphemes from the analysis results, but it does not use the number of types of affixes which can be combined with roots. In [12], removing endings or postpositional particles according to rules was only discussed in terms of the method's uncertainty. Our proposed method is different in that we use the number of types of affixes which can be combined with roots, and we remove values which are below a threshold from the classification target.…”

Section: B Methods For Finding Word Boundariesmentioning

confidence: 99%

See 1 more Smart Citation

Skip-Gram-KR: Korean Word Embedding for Semantic Clustering

Ihm

Lee

2019

IEEE Access

View full text Add to dashboard Cite

Deep learning algorithms are used in various applications for pattern recognition, natural language processing, speech recognition, and so on. Recently, neural network-based natural language processing techniques use fixed length word embedding. Word embedding is a method of digitizing a word at a specific position into a low-dimensional dense vector with fixed length while preserving the similarity of the distribution of its surrounding words. Currently, the word embedding methods for foreign language are used for Korean words; however, existing word embedding methods are developed for English originally, so they do not reflect the order and structure of the Korean words. In this paper, we propose a word embedding method for Korean, which is called Skip-gram-KR, and a Korean affix tokenizer. Skip-gram-KR creates similar word training data through backward mapping and the two-word skipping method. The experiment results show the proposed method achieved the most accurate performance. INDEX TERMS Word embedding, natural language processing, Korean word embedding, text mining, deep learning, semantic clustering, machine learning.

show abstract

Section: B Methods For Finding Word Boundariesmentioning

confidence: 99%

Section: B Methods For Finding Word Boundariesmentioning

confidence: 99%

Skip-Gram-KR: Korean Word Embedding for Semantic Clustering

Ihm

Lee

2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The proposed bridge damage factor recognition model aimed to classify each tokenized word as a bridge element, damage, cause, or other. Since tokenization is language-specific, this paper briefly introduces characteristics of the Korean language and tokenization method for Korean, which were suggested by Kim et al (2014).…”

Section: Preprocessing and Tokenizingmentioning

confidence: 99%

“…This study constructed a vocabulary list from the words used in bridge inspection reports and tokenized the text data in the reports based on the vocabulary list by applying a corpus-based tokenization process. As suggested by Kim et al (2014), all word candidates were first identified from the text to be tokenized, and likelihoods that these word candidates are complete words were measured, which will be referred to as word score. Among all possible consecutive strings of characters in each blank-separated unit, the string whose word score was the largest was extracted as a word, and the other string of characters was separated from the extracted word.…”

Section: Preprocessing and Tokenizingmentioning

confidence: 99%

“…Combining the cohesion probability and the branching entropy, the likelihood that a given string was a complete word is measured, as suggested in Kim et al (2014). In this process, word scores of all possible word candidates from the text data were measured, and a list of all possible vocabularies was constructed along with their corresponding word scores [Eq.…”

Section: Preprocessing and Tokenizingmentioning

confidence: 99%

See 1 more Smart Citation

Bridge Damage Recognition from Inspection Reports Using NER Based on Recurrent Neural Network with Active Learning

Moon

Chung

Chi

2020

J. Perform. Constr. Facil.

View full text Add to dashboard Cite

A deep understanding of the cause-effect relationship of bridge damages provides an opportunity to design, construct, and maintain bridge structures more effectively. The damage factors (i.e., bridge element, damage, and cause) and their complex relationships can be extracted from bridge inspection reports; however, it is not practical to manually read a considerable number of inspection documents and extract such valuable information. Although existing studies attempted to automatically analyze inspection reports, they require a large amount of human effort for data labeling and model development. To overcome the limitations, the authors propose an efficient information acquisition approach that extracts damage factors and causal relationships from bridge inspection reports. The named entity recognition (NER) model was developed based on a recurrent neural network (RNN) and was trained with the active learning method. In the experiments performed with 1,650 sentences (i.e., 1,300 for training and 350 for testing), the developed model successfully classified categories of text words (i.e., damage factors) and captured their causal relationship with 0.927 accuracy and 0.860 F1 score. Besides, the active learning method could significantly reduce the human effort required for data labeling and model development. The developed model achieved 0.778 F1 score only using 140 sentences, requiring less than an hour for manual labeling. These results meant that the model was able to successfully extract major damage factors and their cause-effect relationships from a set of text sentences with little effort. Consequently, the findings of this study can help field engineers to design, construct, and maintain bridge structures.

show abstract

Advanced Subword Segmentation and Interdependent Regularization Mechanisms for Korean Language Understanding

Kim

Lim

et al. 2019

2019 Third World Conference on Smart Trends in Systems Security and Sustainablity (WorldS4)

View full text Add to dashboard Cite

KR-WordRank : An Unsupervised Korean Word Extraction Method Based on WordRank

Cited by 6 publications

References 14 publications

Skip-Gram-KR: Korean Word Embedding for Semantic Clustering

Skip-Gram-KR: Korean Word Embedding for Semantic Clustering

Bridge Damage Recognition from Inspection Reports Using NER Based on Recurrent Neural Network with Active Learning

Advanced Subword Segmentation and Interdependent Regularization Mechanisms for Korean Language Understanding

Contact Info

Product

Resources

About