A large English–Thai parallel corpus from the web and machine-generated text

Lowphansirikul, Lalita; Polpanumas, Charin; Rutherford, Attapol T.; Nutanong, Sarana

doi:10.1007/s10579-021-09536-6

Cited by 28 publications

(32 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Three common machine learning algorithms. To satisfy the conditions of the independence assumption of the plain Bayesian classifier algorithm, we can ignore the dependencies between Thai characters in the Thai subsets and assume that the existence of each Thai character is independent the others 29,30 . We can construct the plain Bayesian classifier model as shown in Figure 1.…”

Section: Methodsmentioning

confidence: 99%

A study of Thai word separation methods based on machine learning

Xiong

Zhao

Duan

2022

Preprint

View full text Add to dashboard Cite

Thai is an analytic, isolated language where monosyllabic words occupy most of the basic vocabulary, and differences in vocabulary and grammar are distinguished by differences in tone. Traditional dictionary-based matching methods have difficulty constructing a complete dictionary. We use four machine learning algorithms, namely, a plain Bayesian classifier, a decision tree classifier, a support vector machine classifier, and a conditional random field, to slice Thai language texts for multiclass problems. In addition, we use the above four machine learning algorithms for a two-class problem slicing of Thai texts. In this study, we found that first, the accuracy, recall and F1 values of Thai text slicing using the conditional random field algorithm in the multiclass classification problem are much higher than those of the plain Bayesian classifier, decision tree classifier and support vector machine classifier. Second, the accuracy, recall, and F1 values of the plain Bayesian algorithm, J48 decision tree algorithm, and support vector machine algorithm are higher than those of the multiclass problem classification in both classes. The accuracy of the conditional random field was improved, but its recall and F1 values decreased significantly. In summary, it can be seen that the conditional random field has better results in Thai word separation.

show abstract

Section: Methodsmentioning

confidence: 99%

A study of Thai word separation methods based on machine learning

Xiong

Zhao

Duan

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Count and display the position of a single word or lexical chunk in the article or corpus, and show it with a black bar chart on a white background, which is equivalent to hot spot analysis. It intuitively displays the specific position of the text search word, so as to facilitate the researchers to identify the distribution of words or lexical chunks in the article or corpus [26].…”

Section: General Function Modulementioning

confidence: 99%

Numerical Analysis and Optimization of English Reading Corpus for Feature Extraction

Wu¹,

Xiong²,

Hongyi³

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

In order to solve the problem of comparative analysis ability of English reading corpus, meet the needs of the construction of language feature analysis system of English reading corpus, make up for the shortcomings of English reading corpus, and improve the wide application of English, the construction method of English reading corpus is proposed. For a long time, English has been an important communication tool in terms of national politics and final economic communication. Especially with the in-depth development of economic globalization, English plays a vital role in international communication. Therefore, the study of the semantics and features of English texts can provide important information materials for English learning and communication in nonnative English speaking countries. Taking this as the starting point, this paper focuses on the numerical analysis and optimization of English reading corpus based on feature extraction and puts forward a language feature analysis system of English reading corpus. At the same time, feature extraction and text classification are used to improve the comparative analysis ability of English reading corpus. The research shows that the feature extraction oriented English reading corpus can solve the problem of English reading learning and improve the ability of reading and analyzing English articles.

show abstract

“…The global COVID-19 fake news also has a chance to be translated and published in Thai social. The open COVID-19 fake news datasets are also translated to Thai as source dataset using SCB-MT-EN-TH translation by VISTEC.AI [78][79][80] as an external knowledge. The source dataset is used to pre-train those transfer learning models (BERT [72], ULMFiT [73] and GPT [74]).…”

Section: English To Thai Translatingmentioning

confidence: 99%

“…The pre-training Thai COVID-19 models was trained by the 123,762 Thai-translated single texts from source dataset (as described in section 3.2). For the data collection, the global COVID-19 fake (and real) news in English from well-known open datasets: CoAID [25,26], ReCOVery [27,28] and FakeCovid [29,30] were collected and translated to Thai using SCB-MT-EN-TH translation by VISTEC.AI [77][78][79]. To evaluate the quality of English to Thai translation, the BLEU [83] was used and compared to other wellknown Eng-to-Thai machines: AI for Thai by NECTEC [47] and Google Translate.…”

Section: Source Dataset By Machine Translationmentioning

confidence: 99%

The COVID-19 fake news detection in Thai social texts

Mookdarsanit

2021

Bulletin EEI

View full text Add to dashboard Cite

One important obstruction against Thai COVID-19 recovery is fake news shared on social media that is one of the “Artificial Intelligence Open Issues against COVID-19” reported by Montreal.AI. Misinformation spread is one of the main cyber-security threats that should be filtered out as the IDS for maintaining COVID-19 information quality. To detect fake news in Thai texts, Thai-NLP techniques are necessary. This paper proposes a state-of-the-art Thai COVID-19 fake news detection among word relations using transfer learning models. For pre-training from the global open COVID-19 datasets, the source dataset is constructed by English to Thai translating. The novel feature shifting is formulated to enlarge Thai text examples in target dataset. Machine translation can be used for constructing Thai source dataset to cope with the lack of local dataset for future Thai-NLP applications. To lead the knowledge in Thai text understanding forward, feature shifting is a promising accuracy improvement in fine-tuning stage.

show abstract

A large English–Thai parallel corpus from the web and machine-generated text

Cited by 28 publications

References 18 publications

A study of Thai word separation methods based on machine learning

A study of Thai word separation methods based on machine learning

Numerical Analysis and Optimization of English Reading Corpus for Feature Extraction

The COVID-19 fake news detection in Thai social texts

Contact Info

Product

Resources

About