Incremental Online Object Learning in a Vehicular Radar-Vision Fusion Framework

Abstract

Abstract-In this report, we propose an object learning system that incorporates sensory information from an automotive radar system and a video camera. The radar system provides a coarse attention for the focus of visual analysis on relatively small areas within the image plane. The attended visual areas are coded and learned by a 3-layer neural network utilizing what is called inplace learning: each neuron is responsible for the learning of its own processing characteristics within the connected network environment, through inhibitory and excitatory connections with other neurons. The modeled bottom-up, lateral, and top-down connections in the network enable sensory sparse coding, unsupervised learning and supervised learning to occur concurrently. The presented work is applied to learn two types of encountered objects in multiple outdoor driving settings. Cross validation results show that the overall recognition accuracy is above 95% for the radar-attended window images. In comparison with the uncoded representation and purely unsupervised learning (without top-down connection), the proposed network improves the overall recognition rate by 15.93% and 6.35%, respectively. The proposed system is also compared with other learning algorithms favorably. The result indicates that our learning system is the only one fit for the incremental and online object learning in the real-time driving environment.Index Terms-Intelligent vehicle system, sensor fusion, object learning, biologically inspired neural network, sparse coding.

Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis

“…In [15], monocular vision was used to solve structure from motion, with radar providing probabilities for objects and the ground surface. In [172], a radar-vision online learning framework was utilized for vehicle detection. Stereo vision has been also used in conjunction with radar sensing [173], [174].…”

Hierarchical discriminative sparse coding via bidirectional connections

“…Finally, n enters the third section for long-term adaptation section: µ(n) increases at a rate of 1/r constantly. As discussed in [23], this kind of plasticity scheduling is more suited for practical signals with unknown non-stationary statistics, where the distribution does not follow i.i.d assumption in all the temporal phase.…”

“…To evaluate the discriminative sparse coding model compared to the unsupervised constructive one, we first define the empirical "probability" to evaluate a cell's updating experience across classes, as used in [23], [24] …”

“…When the number of available cells is larger than the number of classes, the cells representing the same class are grouped together, leading to the simpler boundaries for class decision and thus a better classification result as shown later. See [23], [24] for details and more examples. Without the discriminative topdown connections (i.e., α = 0), however, no class-specific cell groups are observed and cells presenting the same class are scattered around the plane (see Fig.…”

A Planning Tool to Assess Advanced Vehicle Sensor Technologies on Traffic Flow, Fuel Economy, and Emissions

“…However, the merit of integrating a VRS in an advanced vehicle is that the VRS operates offers in a wider range of weather and road conditions and produces similar object detection capabilities [77], [79]- [82]. …”