AddressNet: Shift-Based Primitives for Efficient Convolutional Neural Networks

He, Yihui; Liu, Xianggen; Zhong, Huasong; Ma, Yuchun

doi:10.1109/wacv.2019.00134

Cited by 29 publications

(28 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inference Latency: We also evaluate the inference time of our improved VGG-16 Faster R-CNN on a single GTX 1080 Ti GPU with CUDA 8 and CUDNN 6, as it is crucial for resource-limited applications [51,20,23,19,32]. AP AP 50 AP 60 AP 70 AP 80 AP 90 baseline [14] 38 Table 2, our approach only increases 2ms latency on GPU.…”

Section: Soft-nmsmentioning

confidence: 99%

Bounding Box Regression With Uncertainty for Accurate Object Detection

Zhu

Wang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

503

299

View full text Add to dashboard Cite

Figure 1: In object detection datasets, the ground-truth bounding boxes have inherent ambiguities in some cases. The bounding box regressor is expected to get smaller loss from ambiguous bounding boxes with our KL Loss. (a)(c) The ambiguities introduced by inaccurate labeling. (b) The ambiguities introduced by occlusion. (d) The object boundary itself is ambiguous. It is unclear where the left boundary of the train is because the tree partially occludes it. (better viewed in color) AbstractLarge-scale object detection datasets (e.g., MS-COCO) try to define the ground truth bounding boxes as clear as possible. However, we observe that ambiguities are still introduced when labeling the bounding boxes. In this paper, we propose a novel bounding box regression loss for learning bounding box transformation and localization variance together. Our loss greatly improves the localization accuracies of various architectures with nearly no additional computation. The learned localization variance allows us to merge neighboring bounding boxes during non-maximum suppression (NMS), which further improves the localization performance. On MS-COCO, we boost the Average Precision (AP) of VGG-16 Faster R-CNN from 23.6% to 29.1%. More importantly, for ResNet-50-FPN Mask R-CNN, our method improves the AP and AP 90 by 1.8% and 6.2% respectively, which significantly outperforms previous stateof-the-art bounding box refinement methods. Our code and models are available at github.com/yihui-he/KL-Loss arXiv:1809.08545v3 [cs.CV]

show abstract

Section: Soft-nmsmentioning

confidence: 99%

Bounding Box Regression With Uncertainty for Accurate Object Detection

Zhu

Wang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

503

299

View full text Add to dashboard Cite

show abstract

“…Other works have also investigated varying shifts operations for image classification. Closely related to this work is that of [11], who vary the (discrete) neighbourhood of shifts for miniature images. They then build a compact model for large images (accuracy 67.0%), though the FLOPs and parameters of this model were not shown.…”

Section: Related Workmentioning

confidence: 99%

“…However, active-shifts require additional FLOPs to calculate these interpolations. Further, as activation map motion is non-integer, active shifts always require additional activation map copies in any implementation [11]. [16] also focus on optimising network architectures for miniature image datasets and for computationally constrained models on ImageNet, while we go beyond small datasets and compact networks for shifting.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

4-Connected Shift Residual Networks

Brown

Mettes

Worring

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

The shift operation was recently introduced as an alternative to spatial convolutions. The operation moves subsets of activations horizontally and/or vertically. Spatial convolutions are then replaced with shift operations followed by point-wise convolutions, significantly reducing computational costs. In this work, we investigate how shifts should best be applied to high accuracy CNNs. We apply shifts of two different neighbourhood groups to ResNet on Ima-geNet: the originally introduced 8-connected (8C) neighbourhood shift and the less well studied 4-connected (4C) neighbourhood shift. We find that when replacing ResNet's spatial convolutions with shifts, both shift neighbourhoods give equal ImageNet accuracy, showing the sufficiency of small neighbourhoods for large images. Interestingly, when incorporating shifts to all point-wise convolutions in residual networks, 4-connected shifts outperform 8-connected shifts. Such a 4-connected shift setup gives the same accuracy as full residual networks while reducing the number of parameters and FLOPs by over 40%. We then highlight that without spatial convolutions, ResNet's downsampling/upsampling bottleneck channel structure is no longer needed. We show a new, 4C shift-based residual network, much shorter than the original ResNet yet with a higher accuracy for the same computational cost. This network is the highest accuracy shift-based network yet shown, demonstrating the potential of shifting in deep neural networks.

show abstract

“…FE-Net [4] develops a novel component called Sparse Shift Layer (SSL) to suppress redundant shift operations by adding displacement penalty during optimization. Three shift-based primitives presented in AddressNet [13] significantly reduce the inference time by minimizing the memory copy of shift operations. Meanwhile, the work [21] proposes Temporal Shift Module (TSM) to efficiently model both temporal and spatial information.…”

Section: Shift-based Networkmentioning

confidence: 99%

RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection

Zong

Cao

Leng

2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Feature pyramid networks (FPN) are widely exploited for multiscale feature fusion in existing advanced object detection frameworks. Numerous previous works have developed various structures for bidirectional feature fusion, all of which are shown to improve the detection performance effectively. We observe that these complicated network structures require feature pyramids to be stacked in a fixed order, which introduces longer pipelines and reduces the inference speed. Moreover, semantics from non-adjacent levels are diluted in the feature pyramid since only features at adjacent pyramid levels are merged by the local fusion operation in a sequence manner. To address these issues, we propose a novel architecture named RCNet, which consists of Reverse Feature Pyramid (RevFP) and Cross-scale Shift Network (CSN). RevFP utilizes local bidirectional feature fusion to simplify the bidirectional pyramid inference pipeline. CSN directly propagates representations to both adjacent and non-adjacent levels to enable multi-scale features more correlative. Extensive experiments on the MS COCO dataset demonstrate RCNet can consistently bring significant improvements over both one-stage and two-stage detectors with subtle extra computational overhead. In particular, RetinaNet is boosted to 40.2 AP, which is 3.7 points higher than baseline, by replacing FPN with our proposed model. On COCO test-dev, RCNet can achieve very competitive performance with a single-model single-scale 50.5 AP. Codes will be made available. CCS CONCEPTS• Computing methodologies → Object detection.

show abstract

AddressNet: Shift-Based Primitives for Efficient Convolutional Neural Networks

Cited by 29 publications

References 17 publications

Bounding Box Regression With Uncertainty for Accurate Object Detection

Bounding Box Regression With Uncertainty for Accurate Object Detection

4-Connected Shift Residual Networks

RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection

Contact Info

Product

Resources

About