Spatial-Temporal Fusion Convolutional Neural Network for Simulated Driving Behavior Recognition

Abstract

Abnormal driving behaviour is one of the leading cause of terrible traffic accidents endangering human life. Therefore, study on driving behaviour surveillance has become essential to traffic security and public management. In this paper, we conduct this promising research and employ a two stream CNN framework for video-based driving behaviour recognition, in which spatial stream CNN captures appearance information from still frames, whilst temporal stream CNN captures motion information with pre-computed optical flow displacement between a few adjacent video frames. We investigate different spatial-temporal fusion strategies to combine the intra frame static clues and inter frame dynamic clues for final behaviour recognition. So as to validate the effectiveness of the designed spatial-temporal deep learning based model, we create a simulated driving behaviour dataset, containing 1237 videos with 6 different driving behavior for recognition. Experiment result shows that our proposed method obtains noticeable performance improvements compared to the existing methods.

Moving Object Path Prediction in Traffic Scenes Using Contextual Information

“…Fusion strategies define the way that different streams of features will be joined in the model architecture. Three main fusion strategies are described in [23,24]. Early fusion, mid-level fusion and late fusion.…”

GREENHOME: A Household Energy Consumption and CO 2 Footprint Metering Environment

“…The new values are the estimated actual consumption values grouped by timestamp (see Figure 4). It was necessary to resample from the original data to create datasets, from which the variability of the quantiles of interest could be assessed without long-winded and error-prone analytical calculations [23]. In our experiment, data were sampled in two ways.…”

“…The ARX model is an autoregressive method with exogenous inputs (independent of the process to model) 23 . Autoregressive models express a univariate time series y 𝑛 as a linear combination of past observations y 𝑛−1 and white noise V 𝑛 and are mathematically expressed as [37]:…”

Binary dense sift flow based two stream CNN for human action recognition

“…Human action recognition by these systems is necessary to ensure human safety. The internal driver monitoring system [17] needs to recognize the actions of the driver, and the external monitoring system needs to detect pedestrian [16,22].…”