Beyond core object recognition: Recurrent processes account for object recognition under occlusion

Abstract

Core object recognition, the ability to rapidly recognize objects despite variations in their appearance, is largely solved through the feedforward processing of visual information. Deep neural networks are shown to achieve human-level performance in these tasks, and explain the primate brain representation. On the other hand, object recognition under more challenging conditions (i.e. beyond the core recognition problem) is less characterized. One such example is object recognition under occlusion. It is unclear to what extent feedforward and recurrent processes contribute in object recognition under occlusion. Furthermore, we do not know whether the conventional deep neural networks, such as AlexNet, which were shown to be successful in solving core object recognition, can perform similarly well in problems that go beyond the core recognition. Here, we characterize neural dynamics of object recognition under occlusion, using magnetoencephalography (MEG), while participants were presented with images of objects with various levels of occlusion. We provide evidence from multivariate analysis of MEG data, behavioral data, and computational modelling, demonstrating an essential role for recurrent processes in object recognition under occlusion. Furthermore, the computational model with local recurrent connections, used here, suggests a mechanistic explanation of how the human brain might be solving this problem.

Depth in convolutional neural networks solves scene segmentation

“…To investigate the effect of network depth on scene segmentation, tests were conducted on seven deep residual networks (ResNets; [29] ) with increasing number of layers (6,10,18,34,50,101,152 Wilber [50] was used. In this implementation, input images from the ImageNet dataset [25] were 224x224 randomly cropped from a resized image using the scale and aspect ratio augmentation of Szegedy et al (2015) [51] .…”

“…Models . As in experiment 1, we used deep residual network architectures (ResNets; [29] ) with increasing number of layers (6,10,18,34). We did not use ResNets with more than 34 layers, as the simplicity of the task leads to overfitting problems for the 'ultra-deep' networks.…”

“…Disruption of visual processing beyond feed-forward stages (e.g. >220 ms after stimulus onset, or after activation of higher order areas) can however lead to decreased object recognition performance [4,5] , and a multitude of recent findings suggest that while feed-forward activity may suffice to recognize isolated objects that are easy to discern, the brain employs increasing feedback or recurrent processing for object recognition under more 'challenging' natural conditions [6][7][8][9][10] . When performing a visual object recognition task, the visual input (stimulus) elicits a feed-forward drive that rapidly extracts basic image features through feedforward connections [11] .…”

“…More layers seem especially important when scenes are more difficult or challenging, e.g. because of occlusion, variation, or blurring, where elaborate processing is required [8,10] . The very deep residual networks used in current object recognition tasks are nearly equivalent to a recurrent neural network unfolding over time, when the weights between their hidden layers are clamped [27] .…”

Enhancing multivariate pattern analysis for magnetoencephalography through relevant sensor selection

“…Together with representational similarity analysis, 2 MVPA has demonstrated promising capabilities in revealing subtle neural signature of cognitive processes in magnetoencephalography (MEG), electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data. [3][4][5][6][7][8][9][10][11][12][13][14][15] In many neuroimaging studies, 4,[16][17][18] the pattern vector dimension (number of M/EEG channels or fMRI voxels) highly exceeds the number of data samples (experimental trials), thus incurring the "curse of dimensionality," 19,20 which consequently deteriorates the classifier performance.…”