Computational luminance constancy from naturalistic images

Abstract

The human visual system supports stable percepts of object color even though the light that reflects from object surfaces varies significantly with the scene illumination. To understand the computations that support stable color perception, we study how estimating a target object's luminous reflectance factor (LRF; a measure of the light reflected from the object under a standard illuminant) depends on variation in key properties of naturalistic scenes. Specifically, we study how variation in target object reflectance, illumination spectra, and the reflectance of background objects in a scene impact estimation of a target object's LRF. To do this, we applied supervised statistical learning methods to the simulated excitations of human cone photoreceptors, obtained from labeled naturalistic images. The naturalistic images were rendered with computer graphics. The illumination spectra of the light sources and the reflectance spectra of the surfaces in the scene were generated using statistical models of natural spectral variation. Optimally decoding target object LRF from the responses of a small learned set of task-specific linear receptive fields that operate on a contrast representation of the cone excitations yields estimates that are within 13% of the correct LRF. Our work provides a framework for evaluating how different sources of scene variability limit performance on luminance constancy.

Lightness Discrimination Depends More on Bright Rather Than Shaded Regions of Three-Dimensional Objects

“…We showed this using a physically based rendering system (Heasly, Cottaris, Lichtman, Xiao, & Brainard, 2014; Ward, 1994) to simulate reflections in a large number of three-dimensional realistic shapes. This approach allows to generate a large dataset of object images to statistically relate reflected light to surface properties (e.g., Singh, Cottaris, Heasly, Brainard, & Burge, 2018; Wiebel, Toscani, & Gegenfurtner, 2015). Specifically, we rendered images of different objects, placed in different complex realistic light fields (Debevec, 1998).…”

Equivalent noise characterization of human lightness constancy

“…We are excited about the potential of our paradigm to provide rigorous quantitative insights about the sensory-perceptual processing and the neural computations underlying color constancy in particular, and perceptual constancy more generally. Our previous computational work on lightness constancy is relevant in this context (Singh, Cottaris, Heasly, Brainard, & Burge, 2018).…”

“…We used a statistical model of naturally-occurring surface reflectances to determine the distribution from which we sampled the background surface reflectance functions. This model was developed in our earlier work (Singh, Cottaris, Heasly, Brainard, & Burge, 2018; see also Brainard & Freeman, 1997;Zhang & Brainard, 2004). The model is based on measurements of surface reflectance functions of the Munsell papers (Kelly, Gibson, & Nickerson, 1943) as well as natural surfaces characterized by Vrhel (1994).…”

“…To study the effect of background variation on lightness discrimination thresholds, we varied the reflectance spectra of the background objects in the images by sampling from a statistical model built from natural surface reflectance databases (see Methods: Reflectance and Illumination Spectra Singh, Cottaris, Heasly, Brainard, & Burge, 2018). Briefly, a database of natural surface reflectance functions (Kelly, Gibson, & Nickerson, 1943;Vrhel, Gershon, & Iwan, 1994) was projected along eigenvectors associated with the largest six eigenvalues of the dataset.…”

An image reconstruction framework for characterizing initial visual encoding

“…Such analyses have often been applied to analyze performance for simple artificial stimuli, assuming that the stimuli to be discriminated are known exactly (Banks, Geisler, and Bennett 1987; Davila and Geisler 1991) or known statistically with some uncertainty (Pelli 1985; Geisler 2018). The ideal observer approach has been extended to consider decision processes that learn aspects of the stimuli being discriminated, rather than being provided with these a priori, and extended to handle discrimination and estimation tasks with naturalistic stimuli (Burge and Geisler 2011, 2014; Singh et al 2018; Chin and Burge 2020; Kim and Burge 2020). For a recent review see Burge (2020); also see Tjan and Legge (1998) and Cottaris et al (2019, 2020).…”