Evaluating the robustness of object recognition to visual noise in humans and convolutional neural networks

Jang, Hojin; McCormack, Devin; Tong, Frank

doi:10.1167/17.10.805

Cited by 7 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has also been shown that higher-tier layers of DNNs follow human perceptual shape similarity while the lower-tier layers strictly abide by physical similarity (Kubilius, Bracci, & Op de Beeck, 2016). On the other hand, DNNs are, for instance, much more susceptible to the addition c o r t e x 9 8 ( 2 0 1 8 ) 2 4 9 e2 6 1 of noise to input images than humans (Jang, McCormack, & Tong, 2017) and the exact degree to which the behavior of DNNs and humans overlap is currently a central topic of research.…”

Section: 1mentioning

confidence: 99%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

Deep learning Cost functions RepresentationsVisual processing a b s t r a c t Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural networks. Machine learning has shown that tasks become easier to solve when they are decomposed into subtasks with their own cost function. We hypothesize that the visual system optimizes multiple cost functions of unrelated tasks and this causes the emergence of a ventral pathway dedicated to vision for perception, and a dorsal pathway dedicated to vision for action.To evaluate the functional organization in multi-task deep neural networks, we propose a method that measures the contribution of a unit towards each task, applying it to two networks that have been trained on either two related or two unrelated tasks, using an identical stimulus set. Results show that the network trained on the unrelated tasks shows a decreasing degree of feature representation sharing towards higher-tier layers while the network trained on related tasks uniformly shows high degree of sharing.We conjecture that the method we propose can be used to analyze the anatomical and functional organization of the visual system and beyond. We predict that the degree to which tasks are related is a good descriptor of the degree to which they share downstream cortical-units.

show abstract

Section: 1mentioning

confidence: 99%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

show abstract

“…It has also been shown that higher-tier layers of DNNs follow human perceptual shape similarity while the lower-tier layers strictly abide by physical similarity (Kubilius, Bracci, & Op de Beeck, 2016). On the other hand, DNNs are, for instance, much more susceptible to the addition of noise to input images than humans (Jang, McCormack, & Tong, 2017) and the exact degree to which the behavior of DNNs and humans overlap is currently a central topic of research.…”

Section: Similarities In Architecture and Behavior Between Dnns And T...mentioning

confidence: 99%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2017

Preprint

View full text Add to dashboard Cite

Vision research has been shaped by the seminal insight that we can understand higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural networks. Machine learning has shown that tasks become easier to solve when they are decomposed into subtasks with their own cost function. We hypothesise that the visual system optimizes multiple cost functions of unrelated tasks and this causes the emergence of the ventral pathway, dedicated to vision for perception and dorsal pathway, dedicated to vision for action. To evaluate the functional organization in multi-task deep neural networks we propose a method that measures the contribution of a unit towards each task and apply it to two networks that have been trained on either two related or two unrelated tasks using an identical stimulus set. Results show that the network trained on the unrelated tasks shows a decreasing degree of feature representation sharing towards higher-tier layers while the network trained on related tasks uniformly shows high degree of sharing. We conjecture that the method we propose can be used to reason about the anatomical and functional organization of the visual system and beyond as we predict that the degree to which tasks are related is a good descriptor of the degree to which they can share downstream corticalunits.

show abstract

“…However, other lines of evidence suggest that CNNs are unusually brittle and lack the robustness of human vision, as modest changes in image quality can sometimes lead to catastrophic failure. For example, CNNs can be severely disrupted if a small amount of adversarial noise (Goodfellow, Shlens, & Szegedy, 2014) or a moderate amount of random Gaussian noise (Dodge & Karam, 2017;Geirhos et al, 2018;H Jang, McCormack, & Tong, 2017) is added to the test image. Image blur has likewise been found to impair CNN performance to a degree that would not be expected for human performance (Dodge & Karam, 2017;Geirhos et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

Convolutional neural networks trained with a developmental sequence of blurry to clear images reveal core differences between face and object processing

Jang

Tong

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Although convolutional neural networks (CNNs) provide a promising model for understanding human vision, most CNNs lack robustness to challenging viewing conditions such as image blur, whereas human vision is much more reliable. Might robustness to blur be attributable to vision during infancy, given that acuity is initially poor but improves considerably over the first several months of life? Here, we evaluated the potential consequences of such early experiences by training CNN models on face and object recognition tasks while gradually reducing the amount of blur applied to the training images. For CNNs trained on blurry to clear faces, we observed sustained robustness to blur, consistent with a recent report by Vogelsang and colleagues (2018). By contrast, CNNs trained with blurry to clear objects failed to retain robustness to blur. Further analyses revealed that the spatial frequency tuning of the two CNNs was profoundly different. The blurry to clear face-trained network successfully retained a preference for low spatial frequencies, whereas the blurry to clear object-trained CNN exhibited a progressive shift toward higher spatial frequencies. Our findings provide novel computational evidence showing how face recognition, unlike object recognition, allows for more holistic processing. Moreover, our results suggest that blurry vision during infancy is insufficient to account for the robustness of adult vision to blurry objects.

show abstract

Evaluating the robustness of object recognition to visual noise in humans and convolutional neural networks

Cited by 7 publications

References 0 publications

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Convolutional neural networks trained with a developmental sequence of blurry to clear images reveal core differences between face and object processing

Contact Info

Product

Resources

About