Ultrafast Image Categorization in Biology and Neural Models

Abstract

Humans are able to categorize images very efficiently, in particular to detect the presence of an animal very quickly. Recently, deep learning algorithms based on convolutional neural networks (CNNs) have achieved higher than human accuracy for a wide range of visual categorization tasks. However, the tasks on which these artificial networks are typically trained and evaluated tend to be highly specialized and do not generalize well, e.g., accuracy drops after image rotation. In this respect, biological visual systems are more flexible and efficient than artificial systems for more general tasks, such as recognizing an animal. To further the comparison between biological and artificial neural networks, we re-trained the standard VGG 16 CNN on two independent tasks that are ecologically relevant to humans: detecting the presence of an animal or an artifact. We show that re-training the network achieves a human-like level of performance, comparable to that reported in psychophysical tasks. In addition, we show that the categorization is better when the outputs of the models are combined. Indeed, animals (e.g., lions) tend to be less present in photographs that contain artifacts (e.g., buildings). Furthermore, these re-trained models were able to reproduce some unexpected behavioral observations from human psychophysics, such as robustness to rotation (e.g., an upside-down or tilted image) or to a grayscale transformation. Finally, we quantified the number of CNN layers required to achieve such performance and showed that good accuracy for ultrafast image categorization can be achieved with only a few layers, challenging the belief that image recognition requires deep sequential analysis of visual objects. We hope to extend this framework to biomimetic deep neural architectures designed for ecological tasks, but also to guide future model-based psychophysical experiments that would deepen our understanding of biological vision.

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

“…At present, neuromorphic vision sensors have achieved staged results (Zhou et al, 2019). There are differential visual sampling models that simulate the peripheral sensory motor function of the retina (Jeremie and Perrinet, 2023). There are also integral visual sampling models that simulate the fine texture function of the fovea, such as octopus retina (Purohit and Manohar, 2022), and Vidar (Auge et al, 2021).…”

“…The retina is the first station to receive visual information. The lateral geniculate body is the information transfer station that transmits retinal visual signals to the primary visual cortex; the visual cortex is the visual central processor, which is used in advanced visual functions such as learning and memory, thinking language, and perceptual awareness (Jeremie and Perrinet, 2023). The entire process of visual cortex information processing is completed by two parallel pathways: V1, V2, and V4.…”