Inference via Sparse Coding in a Hierarchical Vision Model

Abstract

Sparse coding has been incorporated in models of the visual cortex for its computational advantages and connection to biology. But how the level of sparsity contributes to performance on visual tasks is not well understood. In this work, sparse coding has been integrated into an existing hierarchical V2 model (Hosoya and Hyvärinen, 2015), but replacing the Independent Component Analysis (ICA) with an explicit sparse coding in which the degree of sparsity can be controlled. After training, the sparse coding basis functions with a higher degree of sparsity resembled qualitatively different structures, such as curves and corners. The contributions of the models were assessed with image classification tasks, including object classification, and tasks associated with mid-level vision including figure-ground classification, texture classification, and angle prediction between two line stimuli. In addition, the models were assessed in comparison to a texture sensitivity measure that has been reported in V2 (Freeman et al., 2013), and a deleted-region inference task. The results from the experiments show that while sparse coding performed worse than ICA at classifying images, only sparse coding was able to better match the texture sensitivity level of V2 and infer deleted image regions, both by increasing the degree of sparsity in sparse coding. Higher degrees of sparsity allowed for inference over larger deleted image regions. The mechanism that allows for this inference capability in sparse coding is described here.

A Sparse Coding Interpretation of Neural Networks and Theoretical Implications

“…While the model lacks convolution, it is trivial to add a convolution-like procedure where sparse inference replaces the dot product of a neural network (see section 4.3). Previous methods of convolutional sparse coding modified the loss function to reconstruct images as a sum of filters convolved with sparse feature maps (Bristow et al, 2013;Wohlberg, 2014), but this loses the original probabilistic interpretation of sparse coding and the associated inference capabilities described by Bowren et al (2021). Instead, this work proposed gathering image patches via a sliding window (like in convolution), but performing sparse inference on each patch.…”

“…The next question evident from this claim is: why would a sparse prior promote proper feature integration (and thus generalization)? The answer to this question is explained by the inductive inference mechanism of sparse coding described by Bowren et al (2021). As the regularization coefficient of sparse coding increases, fewer basis functions must reconstruct the input image.…”

“…These reconstructions have higher error compared to sparse coding with a smaller regularization coefficient, but this error allows the basis functions to introduce their own information into the image representation. Bowren et al (2021) showed that in a hierarchical vision model when a small spatial region of the model's V1 complex cell layer responses was deleted, the missing image information could be inferred with 8x overcomplete non-negative sparse coding (see figures 14 and 15 of their work). In fact, as the value of the regularization coefficient increased, the amount of information able to be inferred also increased.…”

“…The implication is that, given a sufficiently overcomplete representation, large regularization coefficients (and their corresponding priors with high kurtosis) help explain away artefactual information and give the model an understanding of how the basis functions should be integrated to represent image information. Given that the mechanism described by Bowren et al (2021) was studied in non-orthogonal non-negative sparse coding, the general (non-orthogonal) sparse coding transformation may be needed.…”