A Fine-grained Chinese Short Text Classification Method Based on Capsule Networks

Abstract

Chinese short-text classification is an essential primary research direction in natural language processing. The main problem facing the current Chinese short text classification task is how to effectively extract critical semantic feature information and determine the text category within a short text length. Aiming at the problem of feature information extraction of Chinese short texts, this paper introduces a novel sparse attention mechanism. It proposes a Chinese short text classification model RGSC combined with capsule networks. After the model generates word vectors through the RoBERTa pre-training model, the Bi-GRU model initially extracts text feature information. The profound text feature extraction module SC, composed of sparse attention mechanism and capsule networks, is used to further extract critical semantic feature information. The results of ablation experiments on the TouTiao Chinese short text dataset and performance comparison experiments with various models show that the RGSC model can effectively extract the essential text feature information while effectively reducing the irrelevant noise information contained in the text features, and obtain fine-grained text feature information.