Vietnamese Text Classification Algorithm using Long Short Term Memory and Word2Vec

Abstract

In the context of the ongoing forth industrial revolution and fast computer science development the amount of textual information becomes huge. So, prior to applying the seemingly appropriate methodologies and techniques to the above data processing their nature and characteristics should be thoroughly analyzed and understood. At that, automatic text processing incorporated in the existing systems may facilitate many procedures. So far, text classification is one of the basic applications to natural language processing accounting for such factors as emotions’ analysis, subject labeling etc. In particular, the existing advancements in deep learning networks demonstrate that the proposed methods may fit the documents’ classifying, since they possess certain extra efficiency; for instance, they appeared to be effective for classifying texts in English. The thorough study revealed that practically no research effort was put into an expertise of the documents in Vietnamese language. In the scope of our study, there is not much research for documents in Vietnamese. The development of deep learning models for document classification has demonstrated certain improvements for texts in Vietnamese. Therefore, the use of long short term memory network with Word2vec is proposed to classify text that improves both performance and accuracy. The here developed approach when compared with other traditional methods demonstrated somewhat better results at classifying texts in Vietnamese language. The evaluation made over datasets in Vietnamese shows an accuracy of over 90%; also the proposed approach looks quite promising for real applications.

Impact of word embedding models on text analytics in deep learning environment: a review

“…LSTM with the Word2Vec model achieves an F1-score of 98.03% for word segmentation in the Arabic language (Almuhareb et al 2019 ). Neural network-based word embedding efficiently models a word and its context and has become one of the most widely used methods of word distribution representation (N.H. Phat and Anh 2020 )(Alharthi et al 2021 ).…”

“… Malla and Alphonse ( 2021 ) Twitter tweet analysis for the disease information collection COVID-19 labeled English dataset from Twitter Majority voting based ensemble deep learning model RoBERT, BERTweet, CT-BERT RoBERT achieves an accuracy of 90.30% 38. Phat and Anh ( 2020 ) Vietnamese text classification Vietnamese news articles LSTM, CNN, SVM, NB Word2Vec LSTM + Word2Vec achieves an F1-score of 95.74% 39. Grzeça et al ( 2020 ) Social networking site tweets analysis for identification of alcohol-related tweets Datasets DS1-Q1, Q2, Q3 SVM, XGBoost, CNN, BiLSTM DSWE(Drink2Vec), BERT CNN + Drink2Vec achieves an F1-score of 94.45% SANAD Single-label Arabic news articles datasets, NADiA News articles datasets in Arabic with multi-labels, HAN Hierarchical attention network, HDBSCAN Hierarchical Density-Based Spatial Clustering of Applications with Noise, LDA Logistic regression, linear discriminant analysis, QDA Quadratic discriminant analysis, NB Naïve Bayes, SVM Support vector machine, KNN k-nearest neighbor, DT Decision tree, RF Random forest, XGBoost MLP Multilayer perceptron, LIWC Linguistic Inquiry and Word Count features, NER Named entity recognition, PMMC Process model matching contest dataset, DLMF Digital Library of Mathematical Functions, GB Gradient Boosting, SGC Stochastic Gradient Descent, HAN Hierarchical attention network, DFFNN Deep feed-forward neural network.…”