Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines

Doerig, Adrien; Bornet, Alban; Choung, Oh Hyeon; Herzog, Michael H.

doi:10.1101/744268

Cited by 2 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, our results show that, given adequate priors, CapsNets explain uncrowding. We have shown that ffCNNs and CNNs with lateral or top-down recurrent connections do not produce uncrowding, even when they are trained identically on groups of identical shapes and successfully learn on the training data, comparably to the CapsNets (furthermore, we showed previously that ffCNNs trained on large datasets, which are often used as general models of vision, do not show uncrowding either; [17]). This shows that merely training networks on groups of identical shapes is not sufficient to explain uncrowding.…”

Section: Plos Computational Biologymentioning

confidence: 79%

“…Previously, we have shown that these global effects of crowding cannot be explained by models based on the classic framework of vision, including ffCNNs [9,17,38]. Here, we propose a new framework to understand these global effects.…”

Section: Introductionmentioning

confidence: 89%

“…In previous work, we have shown that pretrained ffCNNs cannot explain uncrowding [17], even if they are biased towards global shape processing [13]. Currently, CapsNets cannot be trained on large-scale tasks such as ImageNet because routing by agreement is computationally too expensive.…”

Section: Plos Computational Biologymentioning

confidence: 99%

“…An important point of discussion concerns global visual processing. It was suggested that ffCNNs mainly focus on local, texture-like features, while humans harness global shape computations ( [9,[13][14][15][16][17]; but see [18]). In this context, it was shown that changing local features of an object, such as its texture or edges, leads ffCNNs to misclassify [13,14], while humans can still easily classify the object based on its global shape.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Capsule networks as recurrent models of grouping and segmentation

et al. 2020

Self Cite

View full text Add to dashboard Cite

Classically, visual processing is described as a cascade of local feedforward computations. Feedforward Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. However, using visual crowding as a well-controlled challenge, we previously showed that no classic model of vision, including ffCNNs, can explain human global shape processing. Here, we show that Capsule Neural Networks (CapsNets), combining ffCNNs with recurrent grouping and segmentation, solve this challenge. We also show that ffCNNs and standard recurrent CNNs do not, suggesting that the grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide psychophysical evidence that grouping and segmentation are implemented recurrently in humans, and show that Caps-Nets reproduce these results well. We discuss why recurrence seems needed to implement grouping and segmentation efficiently. Together, we provide mutually reinforcing psychophysical and computational evidence that a recurrent grouping and segmentation process is essential to understand the visual system and create better models that harness global shape computations.

show abstract

Section: Plos Computational Biologymentioning

confidence: 79%

Section: Introductionmentioning

confidence: 89%

Section: Plos Computational Biologymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Capsule networks as recurrent models of grouping and segmentation

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

Capsule Networks as Recurrent Models ofGrouping and Segmentation

Doerig

Schmittwilken

Sayim

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

12Classically, visual processing is described as a cascade of local feedforward computations. Feedforward 13Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. Previously, 14 using visual crowding as a well-controlled challenge, we showed that no classic model of vision, 15including ffCNNs, can explain human global shape processing (1). Here, we show that Capsule Neural 16 Networks (CapsNets; 2), combining ffCNNs with a grouping and segmentation mechanism, solve this 17 challenge. We also show that ffCNNs and standard recurrent networks do not, suggesting that the 18 grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide 19 psychophysical evidence that grouping and segmentation is implemented recurrently in humans, and 20show that CapsNets reproduce these results well. We discuss why recurrence seems needed to 21 implement grouping and segmentation efficiently. Together, we provide mutually reinforcing 22 psychophysical and computational evidence that a recurrent grouping and segmentation process is 23 essential to understand the visual system and create better models that harness global shape 24 computations. 25 26Author Summary 27 Feedforward Convolutional Neural Networks (ffCNNs) have revolutionized computer vision and are 28 deeply transforming neuroscience. However, ffCNNs only roughly mimic human vision. There is a 29 rapidly expanding literature investigating differences between humans and ffCNNs. Several findings 30 suggest that, unlike humans, ffCNNs rely mostly on local visual features. Furthermore, ffCNNs lack 31 recurrent connections, which abound in the brain. Here, we use visual crowding, a well-known 32 psychophysical phenomenon, to investigate recurrent computations in global shape processing. 33Previously, we showed that no model based on the classic feedforward framework of vision, including 34 ffCNNs, can explain global effects in crowding. Here, we show that Capsule Networks (CapsNets), 35combining ffCNNs with recurrent grouping and segmentation, solve this challenge. Lateral and top-36 down recurrent connections do not, suggesting that grouping and segmentation are crucial for 37 human-like global computations. Based on these results, we hypothesize that one computational 38 function of recurrence is to efficiently implement grouping and segmentation. We provide 39 psychophysical evidence that, indeed, recurrent processes implement grouping and segmentation in 40 humans. CapsNets reproduce these results too. Together, we provide mutually reinforcing 41 computational and psychophysical evidence that a recurrent grouping and segmentation process is 42 essential to understand the visual system and create better models that harness global shape 43 computations. 44

show abstract

Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines

Cited by 2 publications

References 32 publications

Capsule networks as recurrent models of grouping and segmentation

Capsule networks as recurrent models of grouping and segmentation

Capsule Networks as Recurrent Models ofGrouping and Segmentation

Contact Info

Product

Resources

About