A Hybrid Classification Method via Character Embedding in Chinese Short Text With Few Words

Abstract

Last decades have witnessed the significance development of research in short text classification. However, most existing methods only focus on the text which contained dozens of words like Twitter or MicroBlog, but not take the short text with few words like news headline or invoice name into consideration. Meanwhile, contemporary short text classification methods either to expand feature of short text with external corpus or to learn the feature representation from all the texts, which have not take the difference between words of short text into full consideration. Notably, the classification of short text with few words are usually determined by a few specific key words contrary to documents classification or traditional short text classification. To address these problems, this paper propose a hybrid classification method of Attention mechanism and Feature selection via Character embedding in Chinese short text with few words, called AFC. More specifically, firstly, the character embedding is computed to represent Chinese short texts with few words, which takes full advantage of short text information without external corpus. Secondly, attention-based LSTM is introduced in our method to project the data into feature representation space with weighting, which make the keywords in classification have more subtle value. Furthermore, the semantic similarity between content and class label information is calculated for feature selection, which reduces the possible negative influence of some redundant information on classification. Experiments on real-world datasets demonstrate the effectiveness of our method compared to other competing methods.

Twenty Years of Machine-Learning-Based Text Classification: A Systematic Review

“…Reference [110] used an end-to-end learning hybrid network with multiple timescales. Other methods, such as character encoding [44], feature expansion [111], and rich feature generation [112] can also improve the performance of a classifier.…”

Impact of word embedding models on text analytics in deep learning environment: a review

“…Word embedding uses neural networks to represent the context and relationships between the target word and its context words (Almuzaini and Azmi 2020 ). An attention mechanism and feature selection using LSTM and character embedding achieve an accuracy of 84.2% in classifying Chinese text (Zhu et al 2020b ). Deep feedforward neural network with the CBOW model achieves an accuracy of 89.56% for fake consumer review detection (Hajek et al 2020 ).…”

“… Shaikh et al ( 2021 ) Bloom's learning outcomes classification Sukkur IBA university dataset, Najran University, Saudi Arabia dataset SVM, NB, LR, RF, RNN, LSTM Word2Vec, fastText, DSWE, GloVe LSTM + DSWE achieves an accuracy of 87% 10. Zhu et al ( 2020b ) Character embedding for Chinese short text classification THUCNews dataset, Toutiao dataset, Invoice dataset RNN, LSTM, HAN Chinese character embedding (AFC) LSTM + AFC achieves an accuracy of 84.2% 11. Roman et al ( 2021 ) Citation Intent Classification Citation Context Dataset, Sci-Cite dataset HDBSCAN GloVe, BERT Kmeans clustering + BERT achieves a precision of 89% 12.…”

“…For the Chinese clinical NER task, the LSTM, CRF, and BERT models obtain an accuracy of 91.60% for EHR categorization. (Li et al 2020b). An LSTM with domain-specific word embedding Tex2Vec is utilized to extract valuable insides from Urdu literature and attain an F1-score of 81.10%.…”