Learnable Gabor modulated complex-valued networks for orientation robustness

Abstract

Robustness to transformation is desirable in many computer vision tasks, given that input data often exhibits pose variance within classes. While translation invariance and equivariance is a documented phenomenon of CNNs, sensitivity to other transformations is typically encouraged through data augmentation. We investigate the modulation of complex valued convolutional weights with learned Gabor filters to enable orientation robustness. With Gabor modulation, the designed network is able to generate orientation dependent features free of interpolation with a single set of rotation-governing parameters. Moreover, by learning rotation parameters alongside traditional convolutional weights, the representation space is not constrained and may adapt to the exact input transformation. We present Learnable Convolutional Gabor Networks (LCGNs), that are parameter-efficient and offer increased model complexity while keeping backpropagation simple. We demonstrate that learned Gabor modulation utilising an end-to-end complex architecture enables rotation invariance and equivariance on MNIST and a new dataset of simulated images of galactic cirri.

Characterization of low surface brightness structures in annotated deep images

“…The annotations we made could be used to train machine learning algorithms. In fact, it has already been the case with Richards et al (2020), who used our cirrus annotations to train a new machine learning algorithm to detect cirrus on deep images. Yet, the small number of annotations is a problem as large datasets are needed to train such algorithms.…”

Multi-Scale Gridded Gabor Attention for Cirrus Segmentation

“…Gabor modulated convolutions have been shown to increase performance on oriented textures and rotated samples e.g. [15,16]. Such layers multiply convolutional weights by Gabor filters with different rotation parameters to generate orientation dependent features.…”

“…Global context is vital in vision: scenes are understood through key descriptive regions, such as grass or sky, as well as through objects. This is especially relevant when processing contaminants covering large regions, such as clouds in remote sensing [7] and solar imaging, [5], or dust clouds in deep sky imaging [16]. Multi-scale (MS) CNNs were proposed to increase global context, e.g.…”

“…Textures are intrinsically composed of orientation dependent patterns, and thus understanding of orientations has been shown to increase performance on a variety of segmentation tasks e.g. [16].…”

“…Additionally, we present a novel attention operator using orientation (Section 2.3), for improved sensitivity to textures. We utilise Gabor modulated convolutions [15,16] to generate features dependent on different angles. Attention is then computed across these angles, measuring correlations between orientation dependent features.…”