Prior knowledge of illumination for 3D perception in the human brain

Abstract

In perceiving 3D shape from ambiguous shading patterns, humans use the prior knowledge that the light is located above their head and slightly to the left. Although this observation has fascinated scientists and artists for a long time, the neural basis of this "light from above left" preference for the interpretation of 3D shape remains largely unexplored. Combining behavioral and functional MRI measurements coupled with multivoxel pattern analysis, we show that activations in early visual areas predict best the light source direction irrespective of the perceived shape, but activations in higher occipitotemporal and parietal areas predict better the perceived 3D shape irrespective of the light direction. These findings demonstrate that illumination is processed earlier than the representation of 3D shape in the visual system. In contrast to previous suggestions, we propose that prior knowledge about illumination is processed in a bottom-up manner and influences the interpretation of 3D structure at higher stages of processing.functional MRI | multi-voxel pattern analysis | shape-from-shading | visual cortex | bottom-up processing A lthough the perception of 3D shape is critically important for actions and interactions in the environments we inhabit, most depth cues are ambiguous. As a result, the brain requires additional information based on previous experience with the environment to infer 3D shape from depth cues. In particular, the inference of 3D shape from shading patterns (i.e., using image luminance intensity variations to derive the shape of a surface) relies on the assumption that the scene illuminant is above our heads and slightly to the left (1, 2). Understanding the illumination of a visual scene has fascinated artists and scientists for a long time (3, 4), but the neural basis of this "light from above left" preference for the interpretation of 3D shape remains largely unexplored. Although the light-from-above preference is consistent with an ecological explanation, the left bias remains entirely unexplained. More generally, the light-from-above preference provides a simple example of the way the brain represents prior knowledge and opens the door for the investigation of other types of prior knowledge related to our perception of the motion, shape, and color of objects.Previous neurophysiological and imaging studies have implicated several brain regions in the processing of shape-fromshading: primary visual cortex (V1) (5-9), areas in the caudal inferior temporal gyrus (10) and the inferior parietal sulcus (11). However, the functions mediated by these different cortical areas may differ. In particular, interpreting shape-from-shading may involve at least two different stages of processing. At the first stage, the contrast polarity of edges in the image (dark or bright) is analyzed and related to the light direction so that left and right light directions can be discriminated. At the second stage, contrast edges are grouped together to form 3D shapes so that convex and concave shapes can be dis...

The Faces of Predictive Coding

“…Activity in this area may reflect deployment of attention to internal mnemonic representations, as stated in the attention to memory hypothesis (AtoM; Wagner et al, 2005). Accordingly, the SMG usually shows decreased BOLD fMRI activity for less familiar information (Wagner et al, 2005;Ciaramelli et al, 2008), potentially corresponding to unusual illumination conditions here. To link these fMRI findings to our MEG results, we draw on the well established positive correlation of the BOLD fMRI signal with GBA in MEG (Brookes et al, 2005).…”

“…PEs are therefore likely to arise in areas involved in the processing 3D shape from shading cues, such as the MFC (Taira et al, 2001). Additionally, SFS may be used to keep shading cues in working memory (Courtney et al, 1998) and ACG may support error detection (Botvinick et al, 2004). Thus, we interpret this illumination effect as a PE signal for the unexpected illumination.…”

Selective responses to specular surfaces in the macaque visual cortex revealed by fMRI

“…Several earlier studies examined the cortical responses to shading in humans (Georgieva et al, 2008;Gerardin et al, 2010;Humphrey et al, 1997;Kourtzi et al, 2003;Moore and Engel, 2001;Taira et al, 2001) and macaques (Arcizet et al, 2009;Hanazawa and Komatsu, 2001;Nelissen et al, 2009), mainly in relation to 3D shape perception from shading. Some of the human fMRI studies reported activation in the higher visual areas, such as the LOC (Kourtzi et al, 2003;Moore and Engel, 2001) and right intraparietal area (Taira et al, 2001), while others also reported response modulation in the early visual areas (Humphrey et al, 1997;Taira et al, 2001), which is consistent with our finding that the early areas responded to surface cues.…”

An empirical assessment of the impact of the light direction on the relief inversion effect in shaded relief maps: NNW is better than NW

“…This is because the convention is not necessarily determined by geography or topography, rather it is fundamentally linked to how the HVS functions. Several studies in perceptual psychology have attempted to measure the overhead illumination bias and observed that, if no other cues are present to indicate the illumination position, the HVS assumes the light to shine not exactly from overhead but from slightly above left, referred to as light-from-aboveleft preference or left bias (Gerardin, Kourtzi, and Mamassian 2010;Mamassian and Goutcher 2001;Sun and Perona 1998). The link between these observations in the perceptual psychology domain and the cartographic convention is not explicitly established in the literature until this point.…”