Approaching ideal observer efficiency in using color to retrieve information from natural scenes

Abstract

Variations in illumination on a scene and trichromatic sampling by the eye limit inferences about scene content. The aim of this work was to elucidate these limits in relation to an ideal observer using color signals alone. Simulations were based on 50 hyperspectral images of natural scenes and daylight illuminants with correlated color temperatures 4000 K, 6500 K, and 25,000 K. Estimates were made of the (Shannon) information available from each scene, the redundancies in receptoral and postreceptoral coding, and the information retrieved by an observer identifying corresponding points across image pairs. For the largest illuminant difference, between 25,000 K and 4000 K, a postreceptoral transformation providing minimum redundancy yielded an efficiency of about 80% in the information retrieved. This increased to about 89% when the transformation was optimized directly for information retrieved, corresponding to an equivalent Gaussian noise amplitude of 3.0% or to a mean of 3.6 x 10(4) distinct identifiable points per scene. Using color signals to retrieve information from natural scenes can approach ideal observer efficiency levels.

Estimating Information from Image Colors: An Application to Digital Cameras and Natural Scenes

“…In one application [11], it was shown that coding at the receptors was highly redundant, as expected given the overlap of the spectral sensitivities of the medium-and long-wavelength-sensitive cone pigments [12]. But with optimal linear postreceptoral processing, redundancy was reduced and efficiency increased so that the information retrieved from color images of natural scenes under different daylight illuminants reached almost 90 percent of that achievable by an ideal observer.…”

“…To provide a reference, the estimators were also applied to the cone photoreceptors of the human eye. Conveniently, mutual information can be interpreted as the logarithm of the mean number of distinct elements or points that can be identified without error across images of the scene under different illuminations [11], [14]. The resulting information estimates therefore provided an answer to the more specific question of the extent to which the elements of a scene can be identified by their colors, independent of spatial position.…”

“…I is the maximum number of distinct points that can be identified reliably across two images of a scene under the two global illuminants e 1 and e 2 [11].…”

“…Some of these disadvantages can be overcome with the use of kernel density estimators [10], [18], but systematic errors in the differential entropy estimates remain [20], [21]. By contrast, methods based on kth-nearest-neighbor statistics [20], [21] avoid estimating pdfs and instead involve calculating distances in the neighborhood of each point in a sample drawn from the spaces spanned by A 1 , A 2 , and A 12 , such as the sets in (10) and (11). The advantages of kth-nearest-neighbor estimators have been documented elsewhere [21], [22], [23].…”

Effectiveness of Digital Camera Sensors in Distinguishing Colored Surfaces in Outdoor Scenes

“…It was used in an offset form that converges more rapidly than the original [2,19]. The noise pdf f W was assumed, as elsewhere [4], to be Gaussian, with covariance matrix K determined by the SNR, which was set nominally to 100 [14].…”