DIANet: Dense-and-Implicit Attention Network

Zhongzhan, Huang,; Liang, Senwei; Liang, Mingfu; Yang, Haizhao

doi:10.1609/aaai.v34i04.5842

Cited by 35 publications

(17 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SGN has also outperformed recent work such as MIM [47], CLS-GAN [48], DSN [49], and BinaryConnect [50]. On the CIFAR-100, SGN has achieved 84.71% outperforming recent studies such as MixMatch [51] Mish [52], DIANet [53], and ResNet-1001 [54]. Table V shows more experimental results on the CIFAR-100.…”

Section: Modelmentioning

confidence: 72%

Signature-Graph Networks

Hamdi

Salim

Kim

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a novel approach for visual representation learning called Signature-Graph Neural Networks (SGN). SGN learns latent global structures that augment the feature representation of Convolutional Neural Networks (CNN). SGN constructs unique undirected graphs for each image based on the CNN feature maps. The feature maps are partitioned into a set of equal and non-overlapping patches. The graph nodes are located on high-contrast sharp convolution features with the local maxima or minima in these patches. The node embeddings are aggregated through novel Signature-Graphs based on horizontal and vertical edge connections. The representation vectors are then computed based on the spectral Laplacian eigenvalues of the graphs. SGN outperforms existing methods of recent graph convolutional networks, generative adversarial networks, and auto-encoders with image classification accuracy of 99.65% on ASIRRA, 99.91% on MNIST, 98.55% on Fashion-MNIST, 96.18% on CIFAR-10, 84.71% on CIFAR-100, 94.36% on STL-10, and 95.86% on SVHN datasets. We also introduce a novel implementation of the state-of-the-art multi-head attention (MHA) on top of the proposed SGN. Adding SGN to MHA improved the image classification accuracy from 86.92% to 94.36% on the STL10 dataset.

show abstract

Section: Modelmentioning

confidence: 72%

Signature-Graph Networks

Hamdi

Salim

Kim

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Through previous work, we found that the results of some models can be effectively improved by introducing low-level features, even for simple aggregations [22]. Beyond that, we inspired by [9] that ConvlSTM is a very powerful module for connecting and integrating multiple layers of information. Therefore, we consider to introduce CONVLSTM module between multi-layer feature map to help feature fusion between different levels.…”

Section: Convlstm For Navigation From High-level To Low-levelmentioning

confidence: 99%

Cross-layer Navigation Convolutional Neural Network for Fine-grained Visual Classification

Guo¹,

Xie²,

Liang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Fine-grained visual classification (FGVC) aims to classify sub-classes of objects in the same super-class (e.g., species of birds, models of cars). For the FGVC tasks, the essential solution is to find discriminative subtle information of the target from local regions. Traditional FGVC models preferred to use the refined features, i.e., high-level semantic information for recognition and rarely use low-level information. However, it turns out that low-level information which contains rich detail information also has effect on improving performance. Therefore, in this paper, we propose cross-layer navigation convolutional neural network for feature fusion. First, the feature maps extracted by the backbone network are fed into a convolutional long short-term memory model sequentially from high-level to low-level to perform feature aggregation. Then, attention mechanisms are used after feature fusion to extract spatial and channel information while linking the high-level semantic information and the low-level texture features, which can better locate the discriminative regions for the FGVC. In the experiments, three commonly used FGVC datasets, including CUB-200-2011, Stanford-Cars, and FGVC-Aircraft datasets, are used for evaluation and we demonstrate the superiority of the proposed method by comparing it with other referred FGVC methods to show that this method achieves superior results.

show abstract

“…First, the added-in module extracts internal information of a networks which can be squeezed channel-wise information (Hu, Shen, and Sun 2018; or spatial information . Next, the module processes the extraction and generates a mask to measure the importance of features via a fully connected layer (Hu, Shen, and Sun 2018), convolution layer (Wang et al 2018) or LSTM (Huang et al 2019b). Last, the mask is applied back to the features to adjust feature importance.…”

Section: Related Workmentioning

confidence: 99%

Instance Enhancement Batch Normalization: An Adaptive Regulator of Batch Noise

Liang

Zhongzhan

Liang³

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

Batch Normalization (BN) (Ioffe and Szegedy 2015) normalizes the features of an input image via statistics of a batch of images and hence BN will bring the noise to the gradient of training loss. Previous works indicate that the noise is important for the optimization and generalization of deep neural networks, but too much noise will harm the performance of networks. In our paper, we offer a new point of view that the self-attention mechanism can help to regulate the noise by enhancing instance-specific information to obtain a better regularization effect. Therefore, we propose an attention-based BN called Instance Enhancement Batch Normalization (IEBN) that recalibrates the information of each channel by a simple linear transformation. IEBN has a good capacity of regulating the batch noise and stabilizing network training to improve generalization even in the presence of two kinds of noise attacks during training. Finally, IEBN outperforms BN with only a light parameter increment in image classification tasks under different network structures and benchmark datasets.

show abstract

DIANet: Dense-and-Implicit Attention Network

Cited by 35 publications

References 19 publications

Signature-Graph Networks

Signature-Graph Networks

Cross-layer Navigation Convolutional Neural Network for Fine-grained Visual Classification

Instance Enhancement Batch Normalization: An Adaptive Regulator of Batch Noise

Contact Info

Product

Resources

About