A CNN-BiGRU Text Classification Model Merging Positional Encoding and Attention Mechanism

Abstract

Aiming at the problems that Convolution Neural Network (CNN) and Bi-directional Gated Recurrent Unit (BiGRU) have insufficient ability to extract text features and cannot obtain the word weight that has a great impact on classification in text, a text classification model based on positional encoding and attention mechanism is proposed. Firstly, positional encoding is introduced before CNN input to obtain the word vector containing text position information, which improves the limited ability of convolution sliding window to extract word association information. At the same time, multichannel CNN and BiGRU are used to obtain local important features and global context information respectively. Finally, the attention mechanism is introduced to fuse the important output features of each channel, and the classification result is obtained by Softmax. The experimental results of two public data sets show that the model can effectively extract text semantic features and improve the accuracy of text classification.