Higher order texture statistics impair contrast boundary segmentation

Arsenault, Elizabeth; Yoonessi, Ahmad; Baker, Curtis L.

doi:10.1167/11.10.14

Cited by 16 publications

(15 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A possible explanation for the improved performance of the L-CAE compared to the baseline linear decoder is that it more fully exploits phase structure that is characteristic of natural images [2], perhaps by incorporating priors on phase structure that are not captured by linear decoding. To test this possibility, we trained both linear and L-CAE decoders on phase-scrambled natural images.…”

Section: Phase Scrambled Trainingmentioning

confidence: 99%

Neural Networks for Efficient Bayesian Decoding of Natural Images from Retinal Neurons

Parthasarathy

Batty

Falcon

et al. 2017

Preprint

View full text Add to dashboard Cite

Decoding sensory stimuli from neural signals can be used to reveal how we sense our physical environment, and is valuable for the design of brain-machine interfaces. However, existing linear techniques for neural decoding may not fully reveal or exploit the fidelity of the neural signal. Here we develop a new approximate Bayesian method for decoding natural images from the spiking activity of populations of retinal ganglion cells (RGCs). We sidestep known computational challenges with Bayesian inference by exploiting artificial neural networks developed for computer vision, enabling fast nonlinear decoding that incorporates natural scene statistics implicitly. We use a decoder architecture that first linearly reconstructs an image from RGC spikes, then applies a convolutional autoencoder to enhance the image. The resulting decoder, trained on natural images and simulated neural responses, significantly outperforms linear decoding, as well as simple point-wise nonlinear decoding. These results provide a tool for the assessment and optimization of retinal prosthesis technologies, and reveal that the retina may provide a more accurate representation of the visual scene than previously appreciated.

show abstract

Section: Phase Scrambled Trainingmentioning

confidence: 99%

Neural Networks for Efficient Bayesian Decoding of Natural Images from Retinal Neurons

Parthasarathy

Batty

Falcon

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Textures were generated on a trial-by-trial basis and subjected to the same homogeneity constraints as described previously (Arsenault, Yoonessi, & Baker, 2011) to preclude spurious luminance or contrast boundaries caused by unevenly distributed micropattern placements. In the low-density condition, only about 12% of the generated textures passed this test; in Figure 1.…”

Section: Texturesmentioning

confidence: 99%

“…Arguably we should use natural texture photographs to begin this investigation to ensure inclusion of any potentially important higher order statistics (Arsenault, Yoonessi, & Baker, 2011). However in pilot tests using boundaries between pairs of natural textures, we observed examples of both improvements and impairments to segmentation thresholds after removing structure by randomizing the phase spectrum, depending on which individual textures were employed.…”

Section: Introductionmentioning

confidence: 99%

Higher order image structure enables boundary segmentation in the absence of luminance or contrast cues

Zavitz

Baker

2014

Journal of Vision

View full text Add to dashboard Cite

Lower order image statistics, which can be described by an image's Fourier energy content, enable segmentation when they are different on either side of a boundary. We have previously demonstrated that the spatial distribution of the energy in an image (described by its higher order statistics or structure) could influence segmentation thresholds for contrast- and orientation-defined boundaries, even though it was the same on either side of the boundary and thus task irrelevant (Zavitz & Baker, 2013). Here we examined whether higher order statistics can also enable segmentation when boundaries are defined by differences in structure or density of texture elements. We used micropattern-based naturalistic synthetic textures to manipulate the sparseness, global phase alignment, and local phase alignment of carrier textures and measured segmentation thresholds based on forced-choice judgments of boundary orientation. We found that both global phase structure and sparseness, but not local phase alignment, enable segmentation and that local structure also has a small effect on segmentation thresholds in both cases. Simulations of a two-stage filter model with a compressive intermediate nonlinearity can reproduce the major features of the experimental data, segmenting boundaries defined by higher order statistics alone while capturing the influence of global image structure on segmentation thresholds.

show abstract

“…For the grayscale patches, we also considered the effects of removing luminance difference cues. However, it is a much harder problem to create luminance-removed patches as was done in previous studies on surface segmentation (Ing et al, 2010), since simply subtracting the mean luminance in each region of an image patch containing an occlusion often yields a high spatial frequency boundary artifact, which provides a strong edge cue (Arsenault et al, 2011). Therefore, we circumvented this problem by setting a 3-pixel thick region around the boundary to the mean luminance of the entire patch, in effect covering up the boundary.…”

Section: Experimental Paradigmmentioning

confidence: 99%

“…In this study, we investigate the question of what locally available cues are used by human subjects to detect occlusion boundaries in natural scenes. We approach this problem by developing a novel database of natural occlusion boundaries taken from a set of uncompressed calibrated images used in previous research (Arsenault, Yoonessi, & Baker, 2011;Kingdom, Field, & Olmos, 2007;Olmos & Kingdom, 2004). We demonstrate that our database exhibits strong intersubject agreement in the locations of the labeled occlusions, particularly when compared with edges derived from image segmentation databases.…”

Section: Introductionmentioning

confidence: 99%

Detecting natural occlusion boundaries using local cues

2012

View full text Add to dashboard Cite

Occlusion boundaries and junctions provide important cues for inferring three-dimensional scene organization from two-dimensional images. Although several investigators in machine vision have developed algorithms for detecting occlusions and other edges in natural images, relatively few psychophysics or neurophysiology studies have investigated what features are used by the visual system to detect natural occlusions. In this study, we addressed this question using a psychophysical experiment where subjects discriminated image patches containing occlusions from patches containing surfaces. Image patches were drawn from a novel occlusion database containing labeled occlusion boundaries and textured surfaces in a variety of natural scenes. Consistent with related previous work, we found that relatively large image patches were needed to attain reliable performance, suggesting that human subjects integrate complex information over a large spatial region to detect natural occlusions. By defining machine observers using a set of previously studied features measured from natural occlusions and surfaces, we demonstrate that simple features defined at the spatial scale of the image patch are insufficient to account for human performance in the task. To define machine observers using a more biologically plausible multiscale feature set, we trained standard linear and neural network classifiers on the rectified outputs of a Gabor filter bank applied to the image patches. We found that simple linear classifiers could not match human performance, while a neural network classifier combining filter information across location and spatial scale compared well. These results demonstrate the importance of combining a variety of cues defined at multiple spatial scales for detecting natural occlusions.

show abstract

Higher order texture statistics impair contrast boundary segmentation

Cited by 16 publications

References 64 publications

Neural Networks for Efficient Bayesian Decoding of Natural Images from Retinal Neurons

Neural Networks for Efficient Bayesian Decoding of Natural Images from Retinal Neurons

Higher order image structure enables boundary segmentation in the absence of luminance or contrast cues

Detecting natural occlusion boundaries using local cues

Contact Info

Product

Resources

About