Toward modeling visual routines of object segmentation with biologically inspired recurrent vision models

Goetschalckx, Lore; Zolfaghar, Maryam; Ashok, Alekh K.; Govindarajan, Lakshmi Narasimhan; Linsley, Drew; Serre, T.

doi:10.1167/jov.22.14.3773

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is an interesting possibility that incorporating such recurrence in deep networks could bring human-like robustness to recognition of objects in unusual poses. Some efforts to incorporate recurrence in deep learning models of the visual system exist (Wyatte et al, 2012;O'Reilly et al, 2013;Spoerer et al, 2017;Tang et al, 2018;Kietzmann et al, 2019;Rajaei et al, 2019;Nayebi et al, 2022;Goetschalckx et al, 2023), but to our knowledge not in the context of recognizing objects in unusual poses. We note that evidence accumulation and recurrence are not mutually exclusive and, in fact, recurrent circuits are one possible neural implementation to allow evidence accumulation.…”

Section: Why Do Humans Need the Extra Time?mentioning

confidence: 99%

A comparison between humans and AI at recognizing objects in unusual poses

Ollikka,

Abbas,

Perin

et al. 2024

Preprint

View full text Add to dashboard Cite

Deep learning is closing the gap with humans on several object recognition benchmarks. Here we investigate this gap in the context of challenging images where objects are seen from unusual viewpoints. We find that humans excel at recognizing objects in unusual poses, in contrast with state-of-the-art pretrained networks (EfficientNet, SWAG, ViT, SWIN, BEiT, ConvNext) which are systematically brittle in this condition. Remarkably, as we limit image exposure time, human performance degrades to the level of deep networks, suggesting that additional mental processes (requiring additional time) take place when humans identify objects in unusual poses. Finally, our analysis of error patterns of humans vs. networks reveals that even time-limited humans are dissimilar to feed-forward deep networks. We conclude that more work is needed to bring computer vision systems to the level of robustness of the human visual system. Understanding the nature of the mental processes taking place during extra viewing time may be key to attain such robustness.

show abstract