CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

Huang, Gao; Liu, Shichen; Maaten, Laurens van der; Weinberger, Kilian Q.

doi:10.1109/cvpr.2018.00291

Cited by 717 publications

(576 citation statements)

References 35 publications

Supporting

Mentioning

575

Contrasting

Order By: Relevance

“…They demonstrated that such sparsity reduces computational cost and number of parameters without compromising accuracy. Huang et al [27] proposed a similar concept, but differs in that the filter groups do not operate on mutually exclusive sets of features. Here we adapt the concept of filter groups to the multi-task learning paradigm and propose an exten-Input (ii) increasing task specialisation (i) uniform splits (iv) other (iii) asymmetrical Figure 4: Illustration of feature routing.…”

Section: Stochastic Filter Groupsmentioning

confidence: 99%

Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels

Bragman

Tanno

Ourselin

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

The performance of multi-task learning in Convolutional Neural Networks (CNNs) hinges on the design of feature sharing between tasks within the architecture. The number of possible sharing patterns are combinatorial in the depth of the network and the number of tasks, and thus hand-crafting an architecture, purely based on the human intuitions of task relationships can be time-consuming and suboptimal. In this paper, we present a probabilistic approach to learning task-specific and shared representations in CNNs for multi-task learning. Specifically, we propose "stochastic filter groups" (SFG), a mechanism to assign convolution kernels in each layer to "specialist" or "generalist" groups, which are specific to or shared across different tasks, respectively. The SFG modules determine the connectivity between layers and the structures of task-specific and shared representations in the network. We employ variational inference to learn the posterior distribution over the possible grouping of kernels and network parameters. Experiments demonstrate that the proposed method generalises across multiple tasks and shows improved performance over baseline methods.

show abstract

Section: Stochastic Filter Groupsmentioning

confidence: 99%

Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels

Bragman

Tanno

Ourselin

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…ShuffleNets [11], [12] utilize low-cost group convolution and channel shuffle. Learning of the group convolution is used across layers in CondenseNet [13]. On the other hand, faster object detections has been achieved in YOLO [14] by introducing a single-stage detection pipeline, where region proposition and classification is performed by one single network simultaneously.…”

Section: Introductionmentioning

confidence: 99%

Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications

Gamanayake

Jayasinghe

et al. 2020

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Even though the Convolutional Neural Networks (CNN) has shown superior results in the field of computer vision, it is still a challenging task to implement computer vision algorithms in real-time at the edge, especially using a low-cost IoT device due to high memory consumption and computation complexities in a CNN. Network compression methodologies such as weight pruning, filter pruning, and quantization are used to overcome the above mentioned problem. Even though filter pruning methodology has shown better performances compared to other techniques, irregularity of the number of filters pruned across different layers of a CNN might not comply with majority of the neural computing hardware architectures. In this paper, a novel greedy approach called cluster pruning has been proposed, which provides a structured way of removing filters in a CNN by considering the importance of filters and the underlying hardware architecture. The proposed methodology is compared with the conventional filter pruning algorithm on Pascal-VOC open dataset, and Head-Counting dataset, which is our own dataset developed to detect and count people entering a room. We benchmark our proposed method on three hardware architectures, namely CPU, GPU, and Intel Movidius Neural Computer Stick (NCS) using the popular SSD-MobileNet and SSD-SqueezeNet neural network architectures used for edge-AI vision applications. Results demonstrate that our method outperforms the conventional filter pruning methodology, using both datasets on above mentioned hardware architectures. Furthermore, a low cost IoT hardware setup consisting of an Intel Movidius-NCS is proposed to deploy an edge-AI application using our proposed pruning methodology.

show abstract

“…The accuracy of those models grows with their complexity, leading to redundant latent representations. Several approaches have been proposed in the literature to reduce this redundancy [8,9,10,11,12], and therefore to improve their efficiency.…”

Section: Introductionmentioning

confidence: 99%

Multi-Scale Octave Convolutions for Robust Speech Recognition

Rownicka

Bell

Renals

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We propose a multi-scale octave convolution layer to learn robust speech representations efficiently. Octave convolutions were introduced by Chen et al [1] in the computer vision field to reduce the spatial redundancy of the feature maps by decomposing the output of a convolutional layer into feature maps at two different spatial resolutions, one octave apart. This approach improved the efficiency as well as the accuracy of the CNN models. The accuracy gain was attributed to the enlargement of the receptive field in the original input space. We argue that octave convolutions likewise improve the robustness of learned representations due to the use of average pooling in the lower resolution group, acting as a low-pass filter. We test this hypothesis by evaluating on two noisy speech corpora -Aurora-4 and AMI. We extend the octave convolution concept to multiple resolution groups and multiple octaves. To evaluate the robustness of the inferred representations, we report the similarity between clean and noisy encodings using an affine projection loss as a proxy robustness measure. The results show that proposed method reduces the WER by up to 6.6% relative for Aurora-4 and 3.6% for AMI, while improving the computational efficiency of the CNN acoustic models.

show abstract

CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

Cited by 717 publications

References 35 publications

Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels

Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels

Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications

Multi-Scale Octave Convolutions for Robust Speech Recognition

Contact Info

Product

Resources

About