Linking patterns of infant eye movements to a neural network model of the ventral stream using representational similarity analysis

Abstract

Little is known about the development of higher-level areas of visual cortex during infancy, and even less is known about how the development of visually guided behavior is related to the different levels of the cortical processing hierarchy. As a first step toward filling these gaps, we used representational similarity analysis (RSA) to assess links between gaze patterns and a neural network model that captures key properties of the ventral visual processing stream. We recorded the eye movements of 4-to 12month-old infants (N = 54) as they viewed photographs of scenes. For each infant, we calculated the similarity of the gaze patterns for each pair of photographs. We also analyzed the images using a convolutional neural network model in which the successive layers correspond approximately to the sequence of areas along the ventral stream.For each layer of the network, we calculated the similarity of the activation patterns for each pair of photographs, which was then compared with the infant gaze data.We found that the network layers corresponding to lower-level areas of visual cortex accounted for gaze patterns better in younger infants than in older infants, whereas the network layers corresponding to higher-level areas of visual cortex accounted for gaze patterns better in older infants than in younger infants. Thus, between 4 and 12 months, gaze becomes increasingly controlled by more abstract, higher-level representations. These results also demonstrate the feasibility of using RSA to link infant gaze behavior to neural network models.

Visual object categorization in infancy

“…Future studies should also address whether other categories can be captured in the infants’ looking behavior by changing the category boundaries (e.g., more/less homogeneous categories), adding other real-world features (e.g., motion), or giving infants more time to explore the images. More, or finer-grained, categories could also be uncovered by going beyond the unidimensional characterization of the infants’ looking behavior afforded by looking times ( 87 ), or replicating the current methodology using neural correlates of infants’ categorization. Finally, the exact nature of the category-relevant visual features that drove DLTs in infants remains to be studied.…”

Perception of an object’s global shape is best described by a model of skeletal structure in human infants

“…Nevertheless, we would not argue that image similarity plays no role in object perception, particularly in infants. Indeed, recent studies comparing the visual representations of infants and computational models reveal that low-level visual similarity explains more variance in infants’ behavioral and neural responses than the upper layers of ANNs ( Kiat et al, 2022 ; Xie et al, 2021 ). Moreover, recent studies suggest that object categorization in infancy may be supported by the representations of the early visual cortex (V1-V3), rather than the higher-level ventral cortex, as in adults ( Spriet et al, 2022 ).…”

“…Importantly, because none of these models represent the shape skeleton, they make for an excellent contrast to the Skeletal model. Finally, we also included a model of pixel similarity, and FlowNet, a model of optic flow ( Ilg et al, 2017 ) in order to assess the extent to which shape representations may be supported by lower-level visual properties like image similarity ( Kiat et al, 2022 ; Xie et al, 2021 ) or motion trajectory ( Kellman, 1984 ; Kellman and Short, 1987 ; Wood and Wood, 2018 ). Altogether, these comparisons provided a novel approach to understanding object perception in human infants.…”

Development of object recognition

“…Indeed, comparisons between infants and ANN models of the ventral visual pathway show that young infants' category representations (4-6 months) are better explained by early layers of the models, which represent simple visual features (Kiat et al, 2021;Xie et al, 2021), than by later layers which are predictive of adult category representations (Blauch, Behrmann, & Plaut, 2022;Rajalingham et al, 2018;Yamins et al, 2014). However, with increased age, infants' category representations become increasingly better described by higher-level layers of the models (Kiat et al, 2021), with the category representation of 18-month-olds being predicted by the multivariate response of adult OTC (Spriet, Abassi, Hochmann, & Papeo, 2021). Thus, a recurring pattern in human development is that many object recognition abilities are present from early in development, but may be supported by different mechanisms.…”