A Novel BGCapsule Network for Text Classification

Abstract

Several text classification tasks such as sentiment analysis, news categorization, multi-label classification and opinion classification are challenging problems even for modern deep learning networks. Recently, Capsule Networks (CapsNets) are proposed for image classification. It has been shown that CapsNets have several advantages over Convolutional Neural Networks (CNNs), while their validity in the domain of text has been less explored. In this paper, we propose a novel hybrid architecture viz., BGCapsule, which is a Capsule model preceded by an ensemble of Bidirectional Gated Recurrent Units (BiGRU) for several text classification tasks. We employed an ensemble of Bidirectional GRUs for feature extraction layer preceding the primary capsule layer. The hybrid architecture, after performing basic pre-processing steps, consists of five layers: an embedding layer based on GloVe, a BiGRU based ensemble layer, a primary capsule layer, a flatten layer and fully connected ReLU layer followed by a fully connected softmax layer. In order to evaluate the effectiveness of BGCapsule, we conducted extensive experiments on five benchmark datasets (ranging from 10,000 records to 700,000 records) including Movie Review (MR Imdb 2005), AG's News dataset, Dbpedia ontology dataset, Yelp Review Full dataset and Yelp review polarity dataset. These benchmarks cover several text classification tasks such as news categorization, sentiment analysis, multiclass classification, multi-label classification and opinion classification. We found that our proposed architecture (BGCapsule) achieves better accuracy compared to the existing methods without the help of any external linguistic knowledge such as positive sentiment keywords and negative sentiment keywords. Further, BGCapsule converged faster compared to other extant techniques.

Evaluation of the Effectiveness of Neural Network Models for Analyzing Customer Review Sentiments on Marketplace

“…LSTM is a universal RNN, when the time span is much longer than the size known with important events, this algorithm is suitable for learning from experience to classify, process, and predict time series data [24]. LSTM consists of three gates: the input gate, the forget gate, and the output gate.…”

BERT-Based Hybrid RNN Model for Multi-class Text Classification to Study the Effect of Pre-trained Word Embeddings

“…Similarly, CNN is combined with GRU layers as an ensemble model to perform MTC on news sources by John et al [18], to help women select the state they want to travel or relocate to, based on the recent criminal activities. As a better alternative to CNN, CapsNets are used along with Bi-GRU layers as a hybrid model, using Word2Vec technique to perform Text classification by Gangwar et al [19]. The detection of fake news also is a significant sub-task of MTC, where in IulianIlie et al [20] have proposed a comparative study of 10 DNN models using GloVe, Word2Vec and FastText word embedding techniques, in which RCNN performed the best.…”