Domain Adaptation of Thai Word Segmentation Models using Stacked Ensemble

Limkonchotiwat, Peerat; Phatthiyaphaibun, Wannaphong; Sarwar, Raheem; Chuangsuwanich, Ekapol; Nutanong, Sarana

doi:10.18653/v1/2020.emnlp-main.315

Cited by 10 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike English, a white space character is not considered as a reliable word boundary indicator in several languages [21]. Similarly, Urdu does not have consistent word boundary markings.…”

Section: ‫ء‬mentioning

confidence: 99%

UrduAI: Writeprints for Urdu Authorship Identification

SarwarRaheem

HassanSaeed-Ul

2021

ACM Trans. Asian Low-Resour. Lang. Inf. Process.

Self Cite

View full text Add to dashboard Cite

The authorship identification task aims at identifying the original author of an anonymous text sample from a set of candidate authors. It has several application domains such as digital text forensics and information retrieval. These application domains are not limited to a specific language. However, most of the authorship identification studies are focused on English and limited attention has been paid to Urdu. However, existing Urdu authorship identification solutions drop accuracy as the number of training samples per candidate author reduces and when the number of candidate authors increases. Consequently, these solutions are inapplicable to real-world cases. Moreover, due to the unavailability of reliable POS taggers or sentence segmenters, all existing authorship identification studies on Urdu text are limited to the word n-grams features only. To overcome these limitations, we formulate a stylometric feature space, which is not limited to the word n-grams feature only. Based on this feature space, we use an authorship identification solution that transforms each text sample into a point set, retrieves candidate text samples, and relies on the nearest neighbors classifier to predict the original author of the anonymous text sample. To evaluate our solution, we create a significantly larger corpus than existing studies and conduct several experimental studies that show that our solution can overcome the limitations of existing studies and report an accuracy level of 94.03%, which is higher than all previous authorship identification works.

show abstract

“…Unlike English, a white space character is not considered as a reliable word boundary indicator in several languages [21]. Similarly, Urdu does not have consistent word boundary markings.…”

Section: ‫ء‬mentioning

confidence: 99%

UrduAI: Writeprints for Urdu Authorship Identification

SarwarRaheem

HassanSaeed-Ul

2021

ACM Trans. Asian Low-Resour. Lang. Inf. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…• SEFR tokenizer; Stacked Ensemble Filter and Refine tokenizer (engine= best¨) [Limkonchotiwat et al, 2020] based on probablities from CNN-based deepcut [Kittinaradorn et al, 2019] with a vocab size of 92,177 words.…”

Section: Preprocessingmentioning

confidence: 99%

“…The following table shows the performance RoBERTa BASE trained on Wikipedia-only dataset. There are four variations of tokenization including subword-level with SentencePiece [Kudo and Richardson, 2018], wordlevel with PyThaiNLP [Phatthiyaphaibun et al, 2020] dictionary-based tokenizer newmm, subword-level with a CRF-based Thai syllable tokenizer ssg , and stacked-ensemble, word-level tokenizer sefr [Limkonchotiwat et al, 2020]. For the RoBERTa BASE trained on Assorted Thai Texts dataset, we only trained with subword token built with Senten-cePiece [Kudo and Richardson, 2018] due to the limited computational resources.…”

Section: Language Modelingmentioning

confidence: 99%

“…In this report, we describe a language model based on RoBERTa-base architecture and SentencePiece [Kudo and Richardson, 2018] subword tokenizer on 78GB cleaned and deduplicated data from publicly available social media posts, news articles, and other open datasets. We also pretrain four other language models using different tokenizers, namely SentencePiece [Kudo and Richardson, 2018], dictionary-based tokenizer (PyThaiNLP's newmm ), syllable tokenizer (ssg ), and SEFR tokenizer [Limkonchotiwat et al, 2020], on Thai Wikipedia Dump to explore how tokens affect downstream performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

WangchanBERTa: Pretraining transformer-based Thai Language Models

Lowphansirikul¹,

Polpanumas²,

Jantrakulchai³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Transformer-based language models, more specifically BERT-based architectures have achieved state-of-the-art performance in many downstream tasks. However, for a relatively low-resource language such as Thai, the choices of models are limited to training a BERT-based model based on a much smaller dataset or finetuning multi-lingual models, both of which yield suboptimal downstream performance. Moreover, large-scale multi-lingual pretraining does not take into account language-specific features for Thai. To overcome these limitations, we pretrain a language model based on RoBERTa-base architecture on a large, deduplicated, cleaned training set (78GB in total size), curated from diverse domains of social media posts, news articles and other publicly available datasets. We apply text processing rules that are specific to Thai most importantly preserving spaces, which are important chunk and sentence boundaries in Thai before subword tokenization. We also experiment with word-level, syllable-level and SentencePiece tokenization with a smaller dataset to explore the effects on tokenization on downstream performance. Our model wangchanberta-baseatt-spm-uncased trained on the 78.5GB dataset outperforms strong baselines (NBSVM, CRF and ULMFit) and multi-lingual models (XLMR and mBERT) on both sequence classification and token classification tasks in human-annotated, mono-lingual contexts.

show abstract

“…The purpose of a corpus in SA is to train the machine learning models with high accuracy. Unfortunately, most of the corpora available as a resource for SA are in English or other popular languages (Ananiadou et al, 2013;Batista-Navarro et al, 2013;Limkonchotiwat et al, 2020;.…”

mentioning

confidence: 99%

Sentiment analysis for Urdu online reviews using deep learning models

et al. 2021

Self Cite

View full text Add to dashboard Cite

Most existing studies are focused on popular languages like English, Spanish, Chinese, Japanese, and others, however, limited attention has been paid to Urdu despite having more than 60 million native speakers. In this paper, we develop a deep learning model for the sentiments expressed in this under-resourced language. We develop an open-source corpus of 10,008 reviews from 566 online threads on the topics of sports, food, software, politics, and entertainment. The objectives of this work are bifold (a) the creation of a human-annotated corpus for the research of sentiment analysis in Urdu; and (b) measurement of up-to-date model performance using a corpus. For their assessment, we performed binary and ternary classification studies utilizing another model, namely long short-term memory (LSTM), recurrent convolutional neural network (RCNN) Rule-Based, N-gram, support vector machine , convolutional neural network, and LSTM. The RCNN model surpasses standard models with 84.98% accuracy for binary classification and 68.56% accuracy for ternary classification. To facilitate other researchers working in the same domain, we have open-sourced the corpus and code developed for this research.

show abstract

Domain Adaptation of Thai Word Segmentation Models using Stacked Ensemble

Cited by 10 publications

References 13 publications

UrduAI: Writeprints for Urdu Authorship Identification

UrduAI: Writeprints for Urdu Authorship Identification

WangchanBERTa: Pretraining transformer-based Thai Language Models

Sentiment analysis for Urdu online reviews using deep learning models

Contact Info

Product

Resources

About