Perceptual Ability with Real-World Nighttime Scenes: Image-Intensified, Infrared, and Fused-Color Imagery

Abstract

We investigated human perceptual performance allowed by relatively impoverished information conveyed in nighttime natural scenes. We used images of nighttime outdoor scenes rendered in image-intensified low-light visible (i2) sensors, thermal infrared (ir) sensors, and an i2/ir fusion technique with information added. We found that nighttime imagery provides adequate low-level image information for effective perceptual organization on a classification task, but that performance for exemplars within a given object category is dependent on the image type. Overall performance was best with the false-color fused images. This is consistent with the suggestion in the literature that color plays a predominate role in perceptual grouping and segmenting of objects in a scene and supports the suggestion that the addition of color in complex achromatic scenes aids the perceptual organization required for visual search. In the present study, we address the issue of assessment of perceptual performance with alternative night-vision sensors and fusion methods and begin to characterize perceptual organization abilities permitted by the information in relatively impoverished images of complex scenes. Applications of this research include improving night vision, medical, and other devices that use alternative sensors or degraded imagery.

Cognitive Image Fusion and Assessment

“…This may lead to a performance that is even worse compared to single band imagery alone (Sinai et al, 1999a). Experiments have indeed convincingly demonstrated that a false color rendering of fused night-time imagery which resembles natural color imagery significantly improves observer performance and reaction times in tasks that involve scene segmentation and classification (Essock et al, 1999;Sinai et al, 1999b;Toet et al, 1997a;Toet & IJspeert, 2001;Vargo, 1999;White, 1998), whereas color mappings that produce counterintuitive (unnaturally looking) results are detrimental to human performance (Krebs et al, 1998;Toet & IJspeert, 2001;Vargo, 1999). One of the reasons often cited for inconsistent color mapping is a lack of physical color constancy (Vargo, 1999).…”

“…However, until now the human end user has not been involved in the design process and the development of image fusion algorithms to any great extent. Mostly, image fusion algorithms are developed in isolation, and the human end-user is little more than an afterthought, so that separate follow-up evaluation studies are usually required to assess to what extent humans benefit from these methods (Aguilar et al, 1999;Dixon et al, 2005;Dixon et al, 2006a;Dixon et al, 2006b;Essock et al, 1999;Essock et al, 2005;Krebs & Sinai, 2002;Smith et al, 2002;Toet & Franken, 2003;Waxman et al, 2006). Recently has it been realized that the only way to guarantee the ultimate effectiveness of image fusion methods for human observers is to include human evaluation as an integral part of the design process (Muller & Narayanan, 2009).…”

[Paper] Visible-Light Image Synthesis from Infrared Images Using Texture Transfer

“…The absence of color may detract from a suitable comprehension of the scene 1) because an object recognition depends on the stored knowledge of object's chromatic characteristics. False color representation makes the reaction time for comprehension of the scene lesser than that of grayscale version 2) , but an image colorized in a manner totally different from that of the actual scene continues to detract from the scene 3) . Therefore, to synthesize a natural-looking visible-light images from IR images, some color images tend to be used as cues of realistic color appearance.…”

An Exploration of Color Fusion with Multispectral Images for Night Vision Enhancement

“…Image fusion combines multiple-source imagery by integrating complementary data in order to enhance the information apparent in the respective source images, as well as to increase the reliability of interpretation. This results in more accurate data (Keys et al, 1990) and increased utility (Rogers & Wood, 1990;Essock et al, 1999). In addition, it has been reported that fused data provides far more robust aspects of operational performance such as increased confidence, reduced ambiguity, improved reliability and improved classification (Rogers & Wood, 1990;Essock et al, 2001).…”

“…On the other hand, a night vision colorization technique can produce colorized imagery with a naturalistic and stable color appearance by processing multispectral night-vision imagery. Although appropriately false-colored imagery is often helpful for human observers in improving their performance on scene classification, and reaction time tasks (Essock et al, 1999;Waxman et al, 1996), inappropriate color mappings can also be detrimental to human performance (Toet & IJspeert, 2001;Varga, 1999). A possible reason is lack of physical color constancy (Varga, 1999).…”