The Research on Tibetan Text Classification Based on N-Gram Model

Abstract

This Compared with the traditional text classification model, the Tibetan text classification based on N-Gram model has adopted N-Gram model in terms of the level of word. In other words, during the text classification, word segmentation is not required. Also, feature selection and abundant pre-treatment processes are avoided. This paper not only carried out profound research on N-Gram models, but also discusses the selection of parameter N in the model by adopting Naïve Bayes Multinomial classifier.

Machine translation of classical Chinese based on unigram segmentation transformer framework

“…Given that different tokenization methods can impact both training efficiency and accuracy of the model, the following analysis provides a brief overview of the effects of these different tokenization methods. [10] [11] involves grouping N characters together and splitting a sentence into segments of N characters each. It is primarily used for calculating the probability of a sentence.…”