Spectral Skyline Separation: Extended Landmark Databases and Panoramic Imaging

Abstract

Evidence from behavioral experiments suggests that insects use the skyline as a cue for visual navigation. However, changes of lighting conditions, over hours, days or possibly seasons, significantly affect the appearance of the sky and ground objects. One possible solution to this problem is to extract the “skyline” by an illumination-invariant classification of the environment into two classes, ground objects and sky. In a previous study (Insect models of illumination-invariant skyline extraction from UV (ultraviolet) and green channels), we examined the idea of using two different color channels available for many insects (UV and green) to perform this segmentation. We found out that for suburban scenes in temperate zones, where the skyline is dominated by trees and artificial objects like houses, a “local” UV segmentation with adaptive thresholds applied to individual images leads to the most reliable classification. Furthermore, a “global” segmentation with fixed thresholds (trained on an image dataset recorded over several days) using UV-only information is only slightly worse compared to using both the UV and green channel. In this study, we address three issues: First, to enhance the limited range of environments covered by the dataset collected in the previous study, we gathered additional data samples of skylines consisting of minerals (stones, sand, earth) as ground objects. We could show that also for mineral-rich environments, UV-only segmentation achieves a quality comparable to multi-spectral (UV and green) segmentation. Second, we collected a wide variety of ground objects to examine their spectral characteristics under different lighting conditions. On the one hand, we found that the special case of diffusely-illuminated minerals increases the difficulty to reliably separate ground objects from the sky. On the other hand, the spectral characteristics of this collection of ground objects covers well with the data collected in the skyline databases, increasing, due to the increased variety of ground objects, the validity of our findings for novel environments. Third, we collected omnidirectional images, as often used for visual navigation tasks, of skylines using an UV-reflective hyperbolic mirror. We could show that “local” separation techniques can be adapted to the use of panoramic images by splitting the image into segments and finding individual thresholds for each segment. Contrarily, this is not possible for ‘global’ separation techniques.

Linear and nonlinear chromatic integration in the mouse retina

“…As input, we took images from a database 56 , 57 of natural scenes with separate UV and green color channels (Fig. 10b ), and converted the pixel intensities to Weber contrast for each color channel.…”

“…For investigating linear and nonlinear chromatic integration with natural scenes, we constructed models of chromatically linear and nonlinear Off cells and simulated responses of populations of such cells to presentations of natural scenes with UV and green color channels. The UV and green images were taken from the “UV/Green Image Databases” 56 , 57 and show scenes of vegetation and sky, which were simultaneously recorded through a dichroic mirror with two cameras and color filters for green light (peak sensitivity at 500 nm, 70 nm bandwidth) and UV light (peak sensitivity at 350 nm, 50 nm bandwidth). Image resolution was 550 × 300 pixels.…”

Rotation invariant visual processing for spatial memory in insects

“…The combination of wide field of view with low resolution has been demonstrated to be beneficial for visual navigation tasks, providing sufficient information while preventing overfitting to details of the environment [17,18]. It is common for insect photoreceptors to be sensitive to ultraviolet light [19], which has been shown to enhance skyline detection capacity [20][21][22][23] underpinning skyline-based navigation strategies [24,25] and demonstrated to be advantageous for autonomous navigation [21,[26][27][28]. Further, specially adapted photoreceptors in the dorsal rim (upwards facing) area of the eye show sensitivity to linearly polarized light, allowing insects to derive their compass orientation from the pattern of polarized light present in the sky [29][30][31].…”

The problem of home choice in skyline-based homing

“…Such a skyline-based mechanism is very appealing from an insect perspective, because the skyline is invariant under illumination changes as caused by changing lighting and weather conditions [ 26 , 27 ]. Several experimental studies suggest that insects might base their navigational endeavours on skyline cues [ 28 – 30 ], while multiple technical studies could establish the feasibility of this approach both experimentally [ 29 , 31 , 32 ] and theoretically [ 33 – 35 ]. We will thus use the ASV model to systematically investigate the question of home choice.…”