Running Large-Scale Simulations on the Neurorobotics Platform to Understand Vision – The Case of Visual Crowding

Abstract

Traditionally, human vision research has focused on specific paradigms and proposed models to explain very specific properties of visual perception. However, the complexity and scope of modern psychophysical paradigms undermine the success of this approach. For example, perception of an element strongly deteriorates when neighboring elements are presented in addition (visual crowding). As it was shown recently, the magnitude of deterioration depends not only on the directly neighboring elements but on almost all elements and their specific configuration. Hence, to fully explain human visual perception, one needs to take large parts of the visual field into account and combine all the aspects of vision that become relevant at such scale. These efforts require sophisticated and collaborative modeling. The Neurorobotics Platform (NRP) of the Human Brain Project offers a unique opportunity to connect models of all sorts of visual functions, even those developed by different research groups, into a coherently functioning system. Here, we describe how we used the NRP to connect and simulate a segmentation model, a retina model, and a saliency model to explain complex results about visual perception. The combination of models highlights the versatility of the NRP and provides novel explanations for inward-outward anisotropy in visual crowding.

Shrinking Bouma’s window: How to model crowding in dense displays

“…Along the same lines, Manassi et al [ 28 ] showed that elements beyond Bouma’s window can have a strong impact on target discrimination, and that the configuration of elements in the whole visual field determines crowding strength (see also [ 26 , 27 ]). A similar extensive comparison of models showed, once again, that only models that could reproduce these results contained a dedicated grouping stage [ 15 ] (see also [ 16 , 43 , 48 ]). Moreover, Van der Burg et al [ 49 ] showed that crowding in dense displays does not depend on target eccentricity but only on the configuration of the nearest neighbours.…”

“…Indeed, without grouping and segmentation to “rescue” the target from the flankers, all elements within Bouma’s window would decrease performance in those models. Grouping and segmentation seem crucial to explain crowding in general [ 10 , 15 , 44 , 48 ]. Moreover, it is known that texture models and other models based on pooling do not reproduce human grouping and segmentation [ 15 , 16 , 43 , 52 , 53 ].…”

Dissecting (un)crowding

“…Capsule networks and the Laminart model are two-stage models, in which elements are first parsed into different groups, and then interference occurs only within the groups. Capsule networks group elements on the basis of object-level routing by agreement (for details, see Doerig et al, 2020 ; Sabour et al, 2017 ), whereas the Laminart model groups elements on the basis of low-level features (for details, see Francis et al, 2017 ; Bornet et al, 2019 ). The TTM model is a one-stage model that pools many low-level features computed over pooling regions whose size grows with eccentricity (for details, see Rosenholtz et al, 2019 ).…”

“…We simulated the conditions of experiment 1 ( Figure 3 ) with Capsule Networks ( Doerig et al, 2020 , https://github.com/adriendoerig/Capsule-networks-as-recurrent-models-of-grouping-and-segmentation ), the Laminart model ( Doerig, Bornet, et al, 2019 , https://bitbucket.org/albornet/laminart/ ) and the texture tiling model (TTM; Rosenholtz et al, 2019 , https://dspace.mit.edu/handle/1721.1/121152 ). Capsule networks were trained to recognize Verniers, groups of squares, groups of horizontal bars, and groups of vertical bars presented in isolation (i.e., there were only flankers or the Vernier).…”

Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines

“…Specifically, we propose that a flexible 275 recurrent grouping process determines which elements are grouped into an object. In the case with a dedicated recurrent grouping process, which is able to explain why (un)crowding occurs 280 (see also Bornet et al, 2019 the crucial benchmarks targeting principled computational processes. Here, using crowding, we 307 showed a fundamental difference in local vs. global processing between humans and ffCNNs, 308 and suggest that grouping and segmentation are promising additions to make deep neural 309 networks better models of vision.…”