A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor

Abstract

This paper describes a 128 times 128 pixel CMOS vision sensor. Each pixel independently and in continuous time quantizes local relative intensity changes to generate spike events. These events appear at the output of the sensor as an asynchronous stream of digital pixel addresses. These address-events signify scene reflectance change and have sub-millisecond timing precision. The output data rate depends on the dynamic content of the scene and is typically orders of magnitude lower than those of conventional frame-based imagers. By combining an active continuous-time front-end logarithmic photoreceptor with a self-timed switched-capacitor differencing circuit, the sensor achieves an array mismatch of 2.1% in relative intensity event threshold and a pixel bandwidth of 3 kHz under 1 klux scene illumination. Dynamic range is > 120 dB and chip power consumption is 23 mW. Event latency shows weak light dependency with a minimum of 15 mus at > 1 klux pixel illumination. The sensor is built in a 0.35 mum 4M2P process. It has 40times40 mum2 pixels with 9.4% fill factor. By providing high pixel bandwidth, wide dynamic range, and precisely timed sparse digital output, this silicon retina provides an attractive combination of characteristics for low-latency dynamic vision under uncontrolled illumination with low post-processing requirements.

Neuromorphic Systems: Past, Present and Future

“…For example, a time to collision detector [Ind98], and a model of the fly elementary motion detector has been built [HK98], and more recently a depth from motion system [YMW + 06]. One particularly interesting recent advance has been a system which detects intensity changes in an overall brightness independent way, and transmits these serially using AER, thus enabling sensing of rapidly altering visual scenes without the huge data rates implied by the need to process whole frames [LPD08].…”

“…In neuromorphic systems, this greediness takes the form of, for example, continuously processing signals from all the pixels, to produce event based signals which may then be sent down an AER bus (as in [LPD08]). In the auditory domain, the numerous band-passed signals are simultaneously processed to search for onsets or for amplitude modulation, mirroring processing processing the the auditory brainstem [SV97].…”

“…The rapid response of the neuromorphic camera in [LPD08] without the use of large-scale frame technology represent a real step forward. However, neuromorphic auditory systems have yet to prove themselves capable: this may be because they have yet to properly incorporate the brainstem processing in more than very simple way on to the filtering technology.…”

Simultaneous Localization and Mapping for Event-Based Vision Systems

“…The integrated Dynamic Vision Sensor (DVS) [3] is a neurobiologically inspired pixel-wise asynchronous vision sensor that generates elementary events ("pieces of information") only for every temporal illumination change perceived by any pixel (see fig. 1).…”

A Real-Time Event-Based Selective Attention System for Active Vision

“…The conventional machine vision system is composed of a standard color camera interfaced to a workstation for executing machine vision algorithms, while the custom hardware past is composed of hybrid analog/digital Very Large Scale Integration (VLSI) chips that implement real-time models of sensory processing systems and neuromorphic models of spiking neurons and cortical neural networks [16]. In particular the neuromorphic multi-chip system presented in this paper comprises a Selective Attention Chip (SAC [1]) inspired by saliency-based models of attention [18] and a Dynamic Vision Sensor (DVS [20]) inspired by the fast transient pathway of mammalian retinas. The DVS is a low-resolution vision sensor that responds to temporal contrast changes in the sensor's field of view in real-time and is not sensitive to color.…”