Linear‐Time Korean Morphological Analysis Using an Action‐based Local Monotonic Attention Mechanism

Hwang, Hyunsun; Lee, Changki

doi:10.4218/etrij.2018-0456

Cited by 3 publications

(4 citation statements)

References 4 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, Korean morphological analyses have witnessed a diverse range of methodologies [3–31]. The agglutinative nature of the Korean language poses challenges that have inspired researchers to devise innovative solutions, laying the foundation for future investigations.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Transformer‐based reranking for improving Korean morphological analysis systems

Ryu,

Lim,

Kwon

et al. 2024

ETRI Journal

View full text Add to dashboard Cite

This study introduces a new approach in Korean morphological analysis combining dictionary‐based techniques with Transformer‐based deep learning models. The key innovation is the use of a BERT‐based reranking system, significantly enhancing the accuracy of traditional morphological analysis. The method generates multiple suboptimal paths, then employs BERT models for reranking, leveraging their advanced language comprehension. Results show remarkable performance improvements, with the first‐stage reranking achieving over 20% improvement in error reduction rate compared with existing models. The second stage, using another BERT variant, further increases this improvement to over 30%. This indicates a significant leap in accuracy, validating the effectiveness of merging dictionary‐based analysis with contemporary deep learning. The study suggests future exploration in refined integrations of dictionary and deep learning methods as well as using probabilistic models for enhanced morphological analysis. This hybrid approach sets a new benchmark in the field and offers insights for similar challenges in language processing applications.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Several approaches have been suggested for morphological analysis, a critical aspect of Korean language comprehension [3–31]. Typically, when individuals grasp spoken or written language, they try to comprehend it through familiar vocabulary and concepts.…”

Section: Introductionmentioning

confidence: 99%

Transformer‐based reranking for improving Korean morphological analysis systems

Ryu,

Lim,

Kwon

et al. 2024

ETRI Journal

View full text Add to dashboard Cite

show abstract

“…Their approach outperforms several other models on grapheme-to-phoneme conversion, transliteration, and morphological inflection. Monotonic attention has also improved tasks such as summarization (Chung et al, 2020) and morphological analysis (Hwang and Lee, 2020).…”

Section: Related Workmentioning

confidence: 99%

On Biasing Transformer Attention Towards Monotonicity

Rios

Amrhein

Aepli

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Many sequence-to-sequence tasks in natural language processing are roughly monotonic in the alignment between source and target sequence, and previous work has facilitated or enforced learning of monotonic attention behavior via specialized attention functions or pretraining. In this work, we introduce a monotonicity loss function that is compatible with standard attention mechanisms and test it on several sequence-to-sequence tasks: grapheme-to-phoneme conversion, morphological inflection, transliteration, and dialect normalization. Experiments show that we can achieve largely monotonic behavior. Performance is mixed, with larger gains on top of RNN baselines. General monotonicity does not benefit transformer multihead attention, however, we see isolated improvements when only a subset of heads is biased towards monotonic behavior.

show abstract

“…However, it is well-known that transformer is insufficient at processing long sequences in terms of performance and computation speed, and speech signals are usually represented by much longer sequences than the written sentences. There have been many research studies to overcome this drawback, including [9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Fast offline transformer‐based end‐to‐end automatic speech recognition for real‐world applications

Park

2021

ETRI Journal

View full text Add to dashboard Cite

With the recent advances in technology, automatic speech recognition (ASR) has been widely used in real-world applications. The efficiency of converting large amounts of speech into text accurately with limited resources has become more vital than ever. In this study, we propose a method to rapidly recognize a large speech database via a transformer-based end-to-end model.Transformers have improved the state-of-the-art performance in many fields. However, they are not easy to use for long sequences. In this study, various techniques to accelerate the recognition of real-world speeches are proposed and tested, including decoding via multiple-utterance-batched beam search, detecting end of speech based on a connectionist temporal classification (CTC), restricting the CTC-prefix score, and splitting long speeches into short segments. Experiments are conducted with the Librispeech dataset and the real-world Korean ASR tasks to verify the proposed methods. From the experiments, the proposed system can convert 8 h of speeches spoken at real-world meetings into text in less than 3 min with a 10.73% character error rate, which is 27.1% relatively lower than that of conventional systems.

show abstract

Linear‐Time Korean Morphological Analysis Using an Action‐based Local Monotonic Attention Mechanism

Cited by 3 publications

References 4 publications

Transformer‐based reranking for improving Korean morphological analysis systems

Transformer‐based reranking for improving Korean morphological analysis systems

On Biasing Transformer Attention Towards Monotonicity

Fast offline transformer‐based end‐to‐end automatic speech recognition for real‐world applications

Contact Info

Product

Resources

About