Drive2Vec: Multiscale State-Space Embedding of Vehicular Sensor Data

Abstract

With automobiles becoming increasingly reliant on sensors to perform various driving tasks, it is important to encode the relevant CAN bus sensor data in a way that captures the general state of the vehicle in a compact form. In this paper, we develop a deep learning-based method, called Drive2Vec, for embedding such sensor data in a low-dimensional yet actionable form. Our method is based on stacked gated recurrent units (GRUs). It accepts a short interval of automobile sensor data as input and computes a low-dimensional representation of that data, which can then be used to accurately solve a range of tasks. With this representation, we (1) predict the exact values of the sensors in the short term (up to three seconds in the future), (2) forecast the long-term average values of these same sensors, (3) infer additional contextual information that is not encoded in the data, including the identity of the driver behind the wheel, and (4) build a knowledge base that can be used to auto-label data and identify risky states. We evaluate our approach on a dataset collected by Audi, which equipped a fleet of test vehicles with data loggers to store all sensor readings on 2,098 hours of driving on real roads. We show in several experiments that our method outperforms other baselines by up to 90%, and we further demonstrate how these embeddings of sensor data can be used to solve a variety of real-world automotive applications.

Brake Maneuver Prediction – An Inference Leveraging RNN Focus on Sensor Confidence

“…It is noteworthy that accelerator events followed by soft brakes are not of our interest, because in such case, a true positive alarm does not contribute significantly to safer driving. Compared to [17], there are significantly more positive examples in our data set (368 positive examples in around 520 hours vs. 122 positive examples in 2098 hours of driving data). Our hard brake definition is slightly more relaxed to focus not only on extreme events.…”

“…The latest work from Hallac et al [17] takes a different approach than focusing solely on braking detection. From a large number of CAN signals, the authors segment individual events by compactly displaying the signals using a RNN as encoder.…”

“…There are many researches that use neural networks to process CAN-Bus data, e.g. [16], [17]. Most of these works aggregate all signals to one neural net-work, aiming to model complex relations between sensors.…”

When Do Drivers Interact with In-Vehicle Well-being Interventions?

“…The observed CAN-busses contained powertrain (e.g., engine and steering wheel), vehicle dynamics (e.g., speed, acceleration, and GPS), and comfort (e.g., air-condition and seat belt usage) messages. 1 Although we recorded all signals in high frequency (> 1 MHz), our subsequent analyses followed best practices with all data re-sampled to 10 Hz [21]. The system booted as soon as the ignition was on and started the recording after 30 seconds to 1 minute.…”

Deep Multi-Task Learning for Anomalous Driving Detection Using CAN Bus Scalar Sensor Data

“…For example, Taylor et al [10] proposed an LSTM predictor based anomaly detection framework for automobiles based on Controller Area Network (CAN) bus data of an automobile similar to ours. Hallac et al [11] present an embedding approach for driving data called Drive2vec which can be used to encode the identity of the driver. However this approach only complements ours, as our approach can work both with raw data as well as embedded data.…”