A Novel Capsule Based Hybrid Neural Network for Sentiment Classification

Du, Yongping; Zhao, Xiaozheng; He, Meng; Guo, Wenyang

doi:10.1109/access.2019.2906398

Cited by 48 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quite coincidentally, Du et al [20] proposed a similar capsule-based hybrid neural network for short text classification. They used capsule network with attention and CNN, RNN architectures.…”

Section: Related Workmentioning

confidence: 99%

A Novel BGCapsule Network for Text Classification

Gangwar

Ravi

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

Several text classification tasks such as sentiment analysis, news categorization, multi-label classification and opinion classification are challenging problems even for modern deep learning networks. Recently, Capsule Networks (CapsNets) are proposed for image classification. It has been shown that CapsNets have several advantages over Convolutional Neural Networks (CNNs), while their validity in the domain of text has been less explored. In this paper, we propose a novel hybrid architecture viz., BGCapsule, which is a Capsule model preceded by an ensemble of Bidirectional Gated Recurrent Units (BiGRU) for several text classification tasks. We employed an ensemble of Bidirectional GRUs for feature extraction layer preceding the primary capsule layer. The hybrid architecture, after performing basic pre-processing steps, consists of five layers: an embedding layer based on GloVe, a BiGRU based ensemble layer, a primary capsule layer, a flatten layer and fully connected ReLU layer followed by a fully connected softmax layer. In order to evaluate the effectiveness of BGCapsule, we conducted extensive experiments on five benchmark datasets (ranging from 10,000 records to 700,000 records) including Movie Review (MR Imdb 2005), AG's News dataset, Dbpedia ontology dataset, Yelp Review Full dataset and Yelp review polarity dataset. These benchmarks cover several text classification tasks such as news categorization, sentiment analysis, multiclass classification, multi-label classification and opinion classification. We found that our proposed architecture (BGCapsule) achieves better accuracy compared to the existing methods without the help of any external linguistic knowledge such as positive sentiment keywords and negative sentiment keywords. Further, BGCapsule converged faster compared to other extant techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

A Novel BGCapsule Network for Text Classification

Gangwar

Ravi

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

show abstract

“…Capsule-based networks have also been recently used for a variety of applications. For example, they have been applied for natural language processing 30 – 33 , with GANs for image generation 34 , computer vision 35 – 37 or medicine 38 , 39 .…”

Section: Related Workmentioning

confidence: 99%

Efficient-CapsNet: capsule network with self-attention routing

Mazzia

Salvetti

Chiaberge

2021

Sci Rep

151

View full text Add to dashboard Cite

Deep convolutional neural networks, assisted by architectural design strategies, make extensive use of data augmentation techniques and layers with a high number of feature maps to embed object transformations. That is highly inefficient and for large datasets implies a massive redundancy of features detectors. Even though capsules networks are still in their infancy, they constitute a promising solution to extend current convolutional networks and endow artificial visual perception with a process to encode more efficiently all feature affine transformations. Indeed, a properly working capsule network should theoretically achieve higher results with a considerably lower number of parameters count due to intrinsic capability to generalize to novel viewpoints. Nevertheless, little attention has been given to this relevant aspect. In this paper, we investigate the efficiency of capsule networks and, pushing their capacity to the limits with an extreme architecture with barely 160 K parameters, we prove that the proposed architecture is still able to achieve state-of-the-art results on three different datasets with only 2% of the original CapsNet parameters. Moreover, we replace dynamic routing with a novel non-iterative, highly parallelizable routing algorithm that can easily cope with a reduced number of capsules. Extensive experimentation with other capsule implementations has proved the effectiveness of our methodology and the capability of capsule networks to efficiently embed visual representations more prone to generalization.

show abstract

“…In terms of computer vision, CapsNet has been used to detect fake images and videos 10 , recognize human movements, learn time information from spatial information 11 , encode facial actions 12 , and classify hyperspectral images 13 , all of which are based on its recognition of image entity attributes. In natural language processing, Zhang et al 14 using capsule network to extract relationship, Du et al 15 proposed a new hybrid neural network based on emotion classification capsules, and McIntosh et al 16 applied multimodal capsule routing to action video segmentation. In medicine, CapsNet has been used to predict Alzheimer disease 17 , automatically classify apoptosis 18 , identify sign language 19 , and classify brain tumor types 20 .…”

Section: Related Workmentioning

confidence: 99%

DA-CapsNet: dual attention mechanism capsule network

Huang

Zhou

2020

Sci Rep

View full text Add to dashboard Cite

A capsule network (CapsNet) is a recently proposed neural network model with a new structure. The purpose of CapsNet is to form activation capsules. In this paper, our team proposes a dual attention mechanism capsule network (DA-CapsNet). In DA-CapsNet, the first layer of the attention mechanism is added after the convolution layer and is referred to as Conv-Attention; the second layer is added after the PrimaryCaps and is referred to as Caps-Attention. The experimental results show that DA-CapsNet performs better than CapsNet. For MNIST, the trained DA-CapsNet is tested in the testset, the accuracy of the DA-CapsNet is 100% after 8 epochs, compared to 25 epochs for CapsNet. For SVHN, CIFAR10, FashionMNIST, smallNORB, and COIL-20, the highest accuracy of DA-CapsNet was 3.46%, 2.52%, 1.57%, 1.33% and 1.16% higher than that of CapsNet. And the results of image reconstruction in COIL-20 show that DA-CapsNet has a more competitive performance than CapsNet.

show abstract

A Novel Capsule Based Hybrid Neural Network for Sentiment Classification

Cited by 48 publications

References 17 publications

A Novel BGCapsule Network for Text Classification

A Novel BGCapsule Network for Text Classification

Efficient-CapsNet: capsule network with self-attention routing

DA-CapsNet: dual attention mechanism capsule network

Contact Info

Product

Resources

About