On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location

Kayhan, Osman Semih; Gemert, Jan C. van

doi:10.1109/cvpr42600.2020.01428

Cited by 118 publications

(81 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rotation and shift are two commonly used data augmentation manners in vision tasks, and these operations should not alter the final results of the model. In other words, we expect translation-invariance [24] in those tasks. However, the absolute positional encoding used in previous transformers, initially designed to leverage the order of tokens, damages such invariance because it adds unique positional encoding to each patch [6].…”

Section: Cmt Blockmentioning

confidence: 92%

CMT: Convolutional Neural Networks Meet Vision Transformers

Guo¹,

Han²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Vision transformers have been successfully applied to image recognition tasks due to their ability to capture long-range dependencies within an image. However, there are still gaps in both performance and computational cost between transformers and existing convolutional neural networks (CNNs). In this paper, we aim to address this issue and develop a network that can outperform not only the canonical transformers, but also the high-performance convolutional models. We propose a new transformer based hybrid network by taking advantage of transformers to capture long-range dependencies, and of CNNs to model local features. Furthermore, we scale it to obtain a family of models, called CMTs, obtaining much better accuracy and efficiency than previous convolution and transformer based models. In particular, our CMT-S achieves 83.5% top-1 accuracy on ImageNet, while being 14x and 2x smaller on FLOPs than the existing DeiT and EfficientNet, respectively. The proposed CMT-S also generalizes well on CIFAR10 (99.2%), CIFAR100 (91.7%), Flowers (98.7%), and other challenging vision datasets such as COCO (44.3% mAP), with considerably less computational cost.

show abstract

Section: Cmt Blockmentioning

confidence: 92%

CMT: Convolutional Neural Networks Meet Vision Transformers

Guo¹,

Han²,

Wu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Position Information Encoded by Neural Network: Recent studies [15,19] have shown that convolutional neural networks can encode the position information of the input image, mainly caused by the padding operation. There also have been a series of works explicitly leverage the position information for devising different architectures (e.g., Transformer network [38], local relation network [13] and generative adversarial network [43]) or enhancing the performance for various computer vision tasks, such as instance segmentation [30], refer expression segmentation [26], etc.…”

Section: Related Workmentioning

confidence: 99%

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

Tang

Jiang

Xie

et al. 2021

Preprint

View full text Add to dashboard Cite

Previous cycle-consistency correspondence learning methods usually leverage image patches for training. In this paper, we present a fully convolutional method, which is simpler and more coherent to the inference process. While directly applying fully convolutional training results in model collapse, we study the underline reason behind this collapse phenomenon, indicating that the absolute positions of pixels provide a shortcut to easily accomplish cycleconsistence, which hinders the learning of meaningful visual representations. To break this absolute position shortcut, we propose to apply different crops for forward and backward frames, and adopt feature warping to establish correspondence between two crops of a same frame. The former technique enforces the corresponding pixels at forward and back tracks to have different absolute positions, and the latter effectively blocks the shortcuts going between forward and back tracks. In three label propagation benchmarks for pose tracking, face landmark tracking and video object segmentation, our method largely improves the results of vanilla fully convolutional cycle-consistency method, achieving very competitive performance compared with the self-supervised state-of-the-art approaches.

show abstract

“…Anti-aliasing in CNNs [55] increases robustness and accuracy. Reducing border effects in CNNs [19] improves translation equivariance and data efficiency. In this paper, we build on these successes and show the benefits of reducing spectral leakage in CNNs.…”

Section: Signal Processing Benefits For Cnnsmentioning

confidence: 99%

Spectral Leakage and Rethinking the Kernel Size in CNNs

Tömen¹,

Gemert²

2021

Preprint

View full text Add to dashboard Cite

Convolutional layers in CNNs implement linear filters which decompose the input into different frequency bands. However, most modern architectures neglect standard principles of filter design when optimizing their model choices regarding the size and shape of the convolutional kernel. In this work, we consider the well-known problem of spectral leakage caused by windowing artifacts in filtering operations in the context of CNNs. We show that the small size of CNN kernels make them susceptible to spectral leakage, which may induce performance-degrading artifacts. To address this issue, we propose the use of larger kernel sizes along with the Hamming window function to alleviate leakage in CNN architectures. We demonstrate improved classification accuracy over baselines with conventional 3 × 3 kernels, on multiple benchmark datasets including Fashion-MNIST, CIFAR-10, CIFAR-100 and ImageNet, via the simple use of a standard window function in convolutional layers. Finally, we show that CNNs employing the Hamming window display increased robustness against certain types of adversarial attacks. Code available at 1 .

show abstract

On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location

Cited by 118 publications

References 48 publications

CMT: Convolutional Neural Networks Meet Vision Transformers

CMT: Convolutional Neural Networks Meet Vision Transformers

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

Spectral Leakage and Rethinking the Kernel Size in CNNs

Contact Info

Product

Resources

About