Predicting the future trajectories of surrounding agents has become an crucial problem to be solved for the safety of autonomous vehicles. Recent studies based on Long Short Term Memory (LSTM) networks have shown powerful abilities to model social interactions. However, many of these approaches focus on spatial interactions of the neighborhood agents but ignore temporal interactions that accompany spatial interactions. In this paper, we propose a Hierarchical Spatio-Temporal Attention architecture (HSTA), which activates the utilization of spatial interactions with different weights, and jointly considers the temporal interactions across time steps of all agents. More specially, the graph attention mechanism (GAT) is presented to capture spatial interactions, the multi-head attention mechanism (MHA) is conducted to encode temporal correlations of interactions and a state gated fusion (SGF) layer is used to integrate spatial and temporal interactions. We evaluate our proposed method against baselines on both pedestrian and vehicle datasets. The results show that our model is effective and achieves state-of-the-art achievements.

The neuromorphic camera is a brand new vision sensor that has emerged in recent years. In contrast to the conventional frame-based camera, the neuromorphic camera only transmits local pixel-level changes at the time of its occurrence and provides an asynchronous event stream with low latency. It has the advantages of extremely low signal delay, low transmission bandwidth requirements, rich information of edges, high dynamic range etc., which make it a promising sensor in the application of in-vehicle visual odometry system. This paper proposes a neuromorphic in-vehicle visual odometry system using feature tracking algorithm. To the best of our knowledge, this is the first in-vehicle visual odometry system that only uses a neuromorphic camera, and its performance test is carried out on actual driving datasets. In addition, an in-depth analysis of the results of the experiment is provided. The work of this paper verifies the feasibility of in-vehicle visual odometry system using neuromorphic cameras.

No abstract

Abnormal event detection is an important task in research and industrial applications, which has received considerable attention in recent years. Existing methods usually rely on standard frame-based cameras to record the data and process them with computer vision technologies. In contrast, this paper presents a novel neuromorphic vision based abnormal event detection system. Compared to the frame-based camera, neuromorphic vision sensors, such as Dynamic Vision Sensor (DVS), do not acquire full images at a fixed frame rate but rather have independent pixels that output intensity changes (called events) asynchronously at the time they occur. Thus, it avoids the design of the encryption scheme. Since events are triggered by moving edges on the scene, DVS is a natural motion detector for the abnormal objects and automatically filters out any temporally-redundant information. Based on this unique output, we first propose a highly efficient method based on the event density to select activated event cuboids and locate the foreground. We design a novel event-based multiscale spatio-temporal descriptor to extract features from the activated event cuboids for the abnormal event detection. Additionally, we build the NeuroAED dataset, the first public dataset dedicated to abnormal event detection with neuromorphic vision sensor. The NeuroAED dataset consists of four sub-datasets: Walking,

No abstract