<title>Gaze-contingent real-time simulation of arbitrary visual fields</title>

Abstract

We describe an algorithm and software for creating variable resolution displays in real time, contingent upon the direction of gaze. The algorithm takes as input a video sequence and an arbitrary, real-valued, two-dimensional map that specifies a desired amount of filtering (blur) at each pixel location relative to direction of gaze. For each input video image the follow operations are performed: (1) the image is coded as a multi-resolution pyramid, (2) the gaze direction is measured, (3) the resolution map is shifted to the gaze direction, (4) the desired filtering at each pixel location is achieved by interpolating between levels of the pyramid using the resolution map, and (5) the interpolated image is displayed. The transfer function associated with each level of the pyramid is calibrated beforehand so that the interpolation produces exactly the desired amount of filtering at each pixel. This algorithm produces precision, artifactfree displays in 8-bit grayscale or 24-bit color. The software can process live or prerecorded video at over 60 frames per second on ordinary personal computers without special hardware. Direction of gaze for each processed video frame may be taken from an eye-tracker, from a sequence of directions saved on disk, or from another pointing device (such as a mouse). The software is demonstrated by simulating the visual fields of normals and of patients with low vision. We are currently using the software to precisely control retinal stimulation during complex tasks such as extended visual search.

Guiding Eye Movements for Better Communication and Augmented Vision

“…This fits well with our previous analysis of eye movements on high resolution natural videos, which shows that eye movements tend to cluster in rather few (10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20) locations [3]. We then reported on simple experiments that were meant to increase the saliency by a brief gazecontingent presentation of red dots in the periphery.…”

“…If the foveation is adjusted to match the resolution distribution of the retina, the effect is not noticeable for the observer, but the resulting images can be compressed more efficiently because they contain less high-frequency content [14,15]. The current state-ofthe-art algorithm for gaze-contingent spatial filtering of video is due to Perry and Geisler [16]. Unlike previous algorithms, which introduced artifacts in the filtered images, their algorithm produces smooth, artifact-free results.…”

Intensified fuzzy clusters for classifying plant, soil, and residue regions of interest from color images

“…The human vision system also integrates tonal variation, probably making color really inseparable. Foveation of the eye or concentration of the mind on specific scene details probably makes visual information content more manageable for humans (Perry and Geisler, 2002). Red (R), green (G), and blue (B) are useful primary colors with associated intensity image subsets.…”

Look together: using gaze for assisting co-located collaborative search

“…This simulates human visual perception. Its resolution is much higher at the fovea focus than the periphery [18], and hence, Spotlight's opacity gradually fades from fovea to periphery. This is similar to the visual space on focused objects in Table 1.…”