Bidirectional Deep Learning of Context Representation for Joint Word Segmentation and POS Tagging

Boonkwan, Prachya; Supnithi, Thepchai

doi:10.1007/978-3-319-61911-8_17

Cited by 10 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, due to the high ambiguity of Thai language, the success rate of dictionary-based approaches, which achieve around 87-92% F1-score [32], is not satisfactory for many applications. In the last years, machine learning-based approaches for Thai word segmentation task gained increased interest [33], [34]. Deepcut is a very recent Thai word segmentation open source project based on deep learning, more precisely convolutional neural networks.…”

Section: ) Evaluation Tool and Metricsmentioning

confidence: 99%

Word Similarity Datasets for Thai: Construction and Evaluation

2019

View full text Add to dashboard Cite

Distributional semantics in the form of word embeddings are an essential ingredient to many modern natural language processing systems. The quantification of semantic similarity between words can be used to evaluate the ability of a system to perform semantic interpretation. To this end, a number of word similarity datasets have been created for the English language over the last decades. For Thai language few such resources are available. In this work, we create three Thai word similarity datasets by translating and re-rating the popular WordSim-353, SimLex-999 and SemEval-2017-Task-2 datasets. The three datasets contain 1852 word pairs in total and have different characteristics in terms of difficulty, domain coverage, and notion of similarity (relatedness vs. similarity). These features help to gain a broader picture of the properties of an evaluated word embedding model. We include baseline evaluations with existing Thai embedding models, and identify the high ratio of out-of-vocabulary words as one of the biggest challenges in the evaluation process. All datasets, evaluation results, and a tool for easy evaluation of new Thai embedding models are available to the NLP community online.

show abstract

Section: ) Evaluation Tool and Metricsmentioning

confidence: 99%

Word Similarity Datasets for Thai: Construction and Evaluation

2019

View full text Add to dashboard Cite

show abstract

“…POS-tags based on transformationbased approaches (Brill, 1995) are designed to automatically derive the possible rules directly from the corpora. Recent years, researchers are moving forward to use machine learning approaches such as Deep learning (Boonkwan & Supnithi, 2017), Neural Network (Li et al, 2017;Viani et al, 2017) and optimization approach such as ant colony-based algorithm (Othmane, Fraj & Limam, 2017). Ant colony-based algorithms are among the most efficient methods to resolve optimization problems modeled as a graph.…”

Section: Introductionmentioning

confidence: 99%

POS-Taggging Malay Corpus: A Novel Approach Based on Maximum Entropy

Bakar

Khairuddin

Nasrudin

et al. 2018

IJET

View full text Add to dashboard Cite

Jawi and Roman scripts are represented Malay language. In the past, Jawi writings are widely used by the Malay community and foreigners; and it can be seen in the old documents. Old documents face the risk of background damage. In order to preserve this valuable information, there are significant needs to automated Jawi materials. Based on previous literature, POS-tags are known as the first phase in the automated text analysis; and the development of language technologies can barely initiate without this phase. We highlight the existing POS-tags approaches; and suggest the development of Malay Jawi POS-tags using extended ME-based approach on NUWT Corpus. Results have shown that the proposed model yielded a higher accuracy in comparison to the state-of-the-art model.

show abstract

“…The rise of deep learning has led to breakthroughs in many areas. In Thai NLP, open-source projects, such as CutKum (Treeratpituk, 2017) and DeepCut (Kittinaradorn etal., 2017) In recent research, Boonkwan and Supnithi (2017) proposed a bidirectional gated recurrent neural networks model for joint word segmentation and POS tagging. This model does not separate word segmentation and POS tagging into two steps but it is a unified model that learn these two tasks simultaneously.…”

Section: Deep Learning In Thai Natural Language Processingmentioning

confidence: 99%

“…This model does not separate word segmentation and POS tagging into two steps but it is a unified model that learn these two tasks simultaneously. Boonkwan and Supnithi (2017) argue that information for POS level also helps constrain word segmentation.…”

Section: Deep Learning In Thai Natural Language Processingmentioning

confidence: 99%

“…Figure3.1: The dynamic deep network for joint word segmentation and POS tagging tasks (Boonkwan and Supnithi, 2017) n-grams; this model has obtained high f1-score on both word segmentation and POS tagging tasks on the ORCHID corpus. In addition, they also found that their model can also cope with the out-of-vocabulary (OOV) issue.…”

Section: Deep Learning In Thai Natural Language Processingmentioning

confidence: 99%

See 1 more Smart Citation

A robust system for core thai natural language processing technologies

Udomcharoenchaikit¹

View full text Add to dashboard Cite

As the amount of unstructured textual data grows, it becomes increasingly important to build an intelligent system that can process it. Natural Language Processing (NLP) is a technology that allows a computer to exploit human languages to perform tasks. Deep learning models have shown excellent results across fundamental tasks in NLP, such as word segmentation, part-of-speech tagging, and named-entity recognition. However, in many situations, these proposed methods fail to perform well. For an NLP system to be robust, it must address issues such as out-of-vocabulary and spelling-mistakes. This thesis's research goal is to develop NLP models that can handle malformed texts to improve their real-world setting usability. In this thesis, I propose novel models and evaluations that focus on robustness against malformed texts. This dissertation proposes multiple novel training strategies and architectures to improve the robustness against malformed texts. This thesis explores input data manipulation strategies that diversify training data, such as UNK masking and adversarial training. It explores how sub-lexical information can improve the robustness of word embeddings. Furthermore, it examines similarity constraint techniques, such as triplet loss, which constraint the similarity between the original texts and the parallel perturbed texts. I also propose alternative evaluation schemes that reveal the weaknesses of NLP systems by introducing typographical adversarial examples to the test sets. Our adversarial evaluation schemes show that current deep learning models are not robust against misspelled inputs, and they also show that our proposed training strategies and architectures can improve the performance over malformed texts.

show abstract

Bidirectional Deep Learning of Context Representation for Joint Word Segmentation and POS Tagging

Cited by 10 publications

References 9 publications

Word Similarity Datasets for Thai: Construction and Evaluation

Word Similarity Datasets for Thai: Construction and Evaluation

POS-Taggging Malay Corpus: A Novel Approach Based on Maximum Entropy

A robust system for core thai natural language processing technologies

Contact Info

Product

Resources

About