Feedforward and Recurrent Processing in Scene Segmentation: Electroencephalography and Functional Magnetic Resonance Imaging

Abstract

In texture segregation, an example of scene segmentation, we can discern two different processes: texture boundary detection and subsequent surface segregation [Lamme, V. A. F., Rodriguez-Rodriguez, V., & Spekreijse, H. Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cerebral Cortex, 9, 406–413, 1999]. Neural correlates of texture boundary detection have been found in monkey V1 [Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378, 492–496, 1995; Grosof, D. H., Shapley, R. M., & Hawken, M. J. Macaque-V1 neurons can signal illusory contours. Nature, 365, 550–552, 1993], but whether surface segregation occurs in monkey V1 [Rossi, A. F., Desimone, R., & Ungerleider, L. G. Contextual modulation in primary visual cortex of macaques. Journal of Neuroscience, 21, 1698–1709, 2001; Lamme, V. A. F. The neurophysiology of figure ground segregation in primary visual-cortex. Journal of Neuroscience, 15, 1605–1615, 1995], and whether boundary detection or surface segregation signals can also be measured in human V1, is more controversial [Kastner, S., De Weerd, P., & Ungerleider, L. G. Texture segregation in the human visual cortex: A functional MRI study. Journal of Neurophysiology, 83, 2453–2457, 2000]. Here we present electroencephalography (EEG) and functional magnetic resonance imaging data that have been recorded with a paradigm that makes it possible to differentiate between boundary detection and scene segmentation in humans. In this way, we were able to show with EEG that neural correlates of texture boundary detection are first present in the early visual cortex around 92 msec and then spread toward the parietal and temporal lobes. Correlates of surface segregation first appear in temporal areas (around 112 msec) and from there appear to spread to parietal, and back to occipital areas. After 208 msec, correlates of surface segregation and boundary detection also appear in more frontal areas. Blood oxygenation level-dependent magnetic resonance imaging results show correlates of boundary detection and surface segregation in all early visual areas including V1. We conclude that texture boundaries are detected in a feedforward fashion and are represented at increasing latencies in higher visual areas. Surface segregation, on the other hand, is represented in “reverse hierarchical” fashion and seems to arise from feedback signals toward early visual areas such as V1.

Act Quickly, Decide Later: Long-latency Visual Processing Underlies Perceptual Decisions but Not Reflexive Behavior

“…The size of the individual squares of the checkerboard was 2.5°by 2.5°visual arc. Although this results in relatively long presentation times for the texture stimuli compared to earlier work, we cannot expect that the longer presentation times will have a large effect on the processing of textures (compare e.g., Fahrenfort, Scholte, & Lamme, 2007, who have used very brief presentation times, and Scholte et al, 2008, using long presentation times, yielding very similar results).…”

“…All statistics are based on the activity in channel Oz according to the International 10-20 System. This electrode was selected on the basis of previous studies (e.g., Scholte et al, 2008;Caputo & Casco, 1999;Lamme, Van Dijk, & Spekreijse, 1992).…”

“…Visual processing of texture checkerboards requires texture segregation, which is a two-stage process: First, borders of figures are detected, and subsequently, the figures are filled in. Border detection occurs around 80-90 msec, and is likely to be the result of lateral inhibition within cortical areas, whereas figure filling-in can take up to 200 msec and depends on re-entrant processing (Scholte, Jolij, Fahrenfort, & Lamme, 2008;Jehee, Roelfsema, Deco, Murre, & Lamme, 2007;Heinen et al, 2005;Roelfsema, Lamme, Spekreijse, & Bosch, 2002;Caputo & Casco, 1999;Lamme, 1995). This latter stage has been linked to perceptual awareness of texture stimuli, whereas the former stage could be sufficient in order to detect presence of a texture stimulus Heinen et al, 2005;Lamme, 1995Lamme, , 2003Supèr, Lamme, & Spekreijse, 2001).…”

The time course of shape discrimination in the human brain

“…Prior work using transient VEPs to study the segmentation of textures into regions has indicated that a differentiation between uniform and segmented stimuli occurs as early as 90-115 ms (Bach and Meigen, 1992;Caputo et al, 1999;Lamme et al, 1992;Scholte et al, 2006Scholte et al, , 2008. Texture-segmentation-related activity localizes to V1 (Appelbaum et al, 2012;Lamme et al, 1992Lamme et al, , 1998.…”

Role of Feedforward and Feedback Projections in Figure-Ground Responses

“…Feedback accomplishes this by a differential effect on neural activity; it enhances figure responses and lowers background responses (Supèr & Romeo, 2011). Such push-pull effect is also observed in neurons of the visual cortex responding to figure-ground textures (Supèr et al, 2001a;Landman et al, 2003a;Scholte et al, 2008). Moreover, the model shows that feedback especially enhances figure-ground signal when the feedforward input is relatively weak.…”