NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving With a New Database and Baseline Evaluations

Abstract

Neuromorphic vision sensors such as the Dynamic and Active-pixel Vision Sensor (DAVIS) using silicon retina are inspired by biological vision, they generate streams of asynchronous events to indicate local log-intensity brightness changes. Their properties of high temporal resolution, low-bandwidth, lightweight computation, and lowlatency make them a good fit for many applications of motion perception in the intelligent vehicle. However, as a younger and smaller research field compared to classical computer vision, neuromorphic vision is rarely connected with the intelligent vehicle. For this purpose, we present three novel datasets recorded with DAVIS sensors and depth sensor for the distracted driving research and focus on driver drowsiness detection, driver gaze-zone recognition, and driver hand-gesture recognition. To facilitate the comparison with classical computer vision, we record the RGB, depth and infrared data with a depth sensor simultaneously. The total volume of this dataset has 27360 samples. To unlock the potential of neuromorphic vision on the intelligent vehicle, we utilize three popular event-encoding methods to convert asynchronous event slices to event-frames and adapt state-of-the-art convolutional architectures to extensively evaluate their performances on this dataset. Together with qualitative and quantitative results, this work provides a new database and baseline evaluations named NeuroIV in cross-cutting areas of neuromorphic vision and intelligent vehicle.

The Alleviation of Perceptual Blindness During Driving in Urban Areas Guided by Saccades Recommendation

“…In [23], the authors compared several higher-order cognitive abilities involving manipulation and storage of visuospatial information under speeded conditions, and proposed a training intervention to improve driving skills and maintain safe driving. In [24], the neuromorphic vision sensor by mimicking human vision sensitivity to brightness changes is embedded in intelligent vehicles for the improvement of driving safety.…”

Research on Vehicle Automatic Driving Target Perception Technology Based on Improved MSRPN Algorithm

“…The parameters of the surrounding environment are obtained through the on-board camera, to serve automatic driving. [13][14][15][16] As required, the machine vision perception platform for automatic vehicle driving is equipped with 6 cameras, which are arranged according to three different functions: front view, measuring view and rear view. The front camera mainly captures the image information of the road in front of the vehicle, obtains the front environment according to the obtained information, and empties the information for the system's early warning and vehicle control through in-depth analysis.…”

Unconventional Visual Sensors for Autonomous Vehicles

“…For example, leveraging "frequency encoding" representation from Chen et al [168], where a standard YOLOv3 CNN architecture [79] is utilized for pedestrian detection. Chen et al [169] also get the best results with the "frequency encoding" among other encoding schemes for driver monitoring applications. Perot et al [170] test different accumulation and encoding strategies for object detection, with the best results using the "discretized event volume" representation from Zhu et al [171].…”

“…We expect to see more and more HD event camera datasets as Perot et al [170] appear in public. The published datasets are for various purposes, such as target detection [176], lane detection [177], drowsiness detection [169] etc. A comprehensive summary of the current open datasets is demonstrated in Table II.…”

“…Chen et al [190] focus on drowsiness detection, and compare some classification algorithms on their event dataset. Provided with a new dataset, Chen et al [169] compare different CNN architectures and events accumulation schemes for driver drowsiness detection, gaze-zone recognition, and hand-gesture recognition.…”