Training for object recognition with increasing spatial frequency: A comparison of deep learning with human vision

Abstract

The ontogenetic development of human vision and the real-time neural processing of visual input exhibit a striking similarity—a sensitivity toward spatial frequencies that progresses in a coarse-to-fine manner. During early human development, sensitivity for higher spatial frequencies increases with age. In adulthood, when humans receive new visual input, low spatial frequencies are typically processed first before subsequent processing of higher spatial frequencies. We investigated to what extent this coarse-to-fine progression might impact visual representations in artificial vision and compared this to adult human representations. We simulated the coarse-to-fine progression of image processing in deep convolutional neural networks (CNNs) by gradually increasing spatial frequency information during training. We compared CNN performance after standard and coarse-to-fine training with a wide range of datasets from behavioral and neuroimaging experiments. In contrast to humans, CNNs that are trained using the standard protocol are very insensitive to low spatial frequency information, showing very poor performance in being able to classify such object images. By training CNNs using our coarse-to-fine method, we improved the classification accuracy of CNNs from 0% to 32% on low-pass-filtered images taken from the ImageNet dataset. The coarse-to-fine training also made the CNNs more sensitive to low spatial frequencies in hybrid images with conflicting information in different frequency bands. When comparing differently trained networks on images containing full spatial frequency information, we saw no representational differences. Overall, this integration of computational, neural, and behavioral findings shows the relevance of the exposure to and processing of inputs with variation in spatial frequency content for some aspects of high-level object representations.

Why is human vision so poor in early development? The impact of initial sensitivity to low spatial frequencies on visual category learning

“…Under this view, we hypothesize that the underdeveloped state of the newborn human visual is functional and, among other things, facilitates the infant’s rapid acquisition of basic-level categories. The work we present here, along with several related studies (French et al, 2002; Vogelsang et al, 2018; Avberšek et al, 2021; Jang & Tong, 2021), supports this hypothesis, suggesting that poor infant vision – in the form of low contrast sensitivity – confers multiple adaptive benefits to the visual learner.…”

“…Early experience with blurry images may also improve the robustness of visual recognition across image degradation. This is supported by a recent study by Avberšek et al (2021), who used a coarse-to-fine image training regimen with multiple CNN models. More specifically, they attempted to mirror the trajectory of improving contrast sensitivity over early human development by initially training their models only with lower spatial frequency filtered images and then gradually introducing higher spatial frequencies as training progressed.…”

“…Finally, several recent studies have also investigated how training with blurred images impacts learning in neural network models Avberšek et al (2021). investigated the effects of training a neural network on various regimens of progressively blurry-to-clear images.…”

Do training with blurred images make convolutional neural networks closer to humans concerning object recognition performance and internal representations?

“…face classification) [Jang and Tong, 2021]. A recent study using object recognition [Avberšek et al, 2021] reports the effect of training schedule consistent with ours. The task difference may be related to the fact that the optimal discriminative features for object recognition are biased toward high frequencies while only low-frequency features are sufficient for good face classification accuracy [Jang and Tong, 2021].…”

“…It has been suggested that the experience of blurred visual images might be functionally beneficial, enabling the visual system to use global configural structures in image recognition [Grand et al, 2001, Le Grand et al, 2004, Vogelsang et al, 2018. Several recent studies use machine learning of artificial neural networks to computationally test this hypothesis [Vogelsang et al, 2018, Katzhendler and Weinshall, 2019, Avberšek et al, 2021, Jang and Tong, 2021. Vogelsang trained the convolutional network (CNN) to recognize human faces.…”