Dynamic representations for autonomous driving

Olier, Juan Sebastian; Marin-Plaza, Pablo; Martín, David; Barakova, Emilia; Rauterberg, Matthias; Regazzoni, Carlo S.

doi:10.1109/avss.2017.8078511

Cited by 8 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works have been presented in the context of anomaly detection of autonomous systems. Olier et al [21] demonstrated how an autonomous agent can be trained to mimic human behavior using a variational deep generative architecture, and Baydoun et al [22] introduced an anomaly detection model by fusing two viewpoints together; the shared layer and the private layer. Here, the shared layer observes odometry data of all the moving agents externally while the private layer is only accessible to the relevant agent and observes visual data captured by each agent.…”

Section: Related Workmentioning

confidence: 99%

Anomaly Detection using Deep Reconstruction and Forecasting for Autonomous Systems

Bahavan,

Suman,

Cader

et al. 2020

Preprint

View full text Add to dashboard Cite

We propose self-supervised deep algorithms to detect anomalies in heterogeneous autonomous systems using frontal camera video and IMU readings. Given that the video and IMU data are not synchronized, each of them are analyzed separately. The vision-based system, which utilizes a conditional GAN, analyzes immediate-past three frames and attempts to predict the next frame. The frame is classified as either an anomalous case or a normal case based on the degree of difference estimated using the prediction error and a threshold. The IMUbased system utilizes two approaches to classify the timestamps; the first being an LSTM autoencoder which reconstructs three consecutive IMU vectors and the second being an LSTM forecaster which is utilized to predict the next vector using the previous three IMU vectors. Based on the reconstruction error, the prediction error, and a threshold, the timestamp is classified as either an anomalous case or a normal case. The composition of algorithms won runners up at the IEEE Signal Processing Cup anomaly detection challenge 2020. In the competition dataset of camera frames consisting of both normal and anomalous cases, we achieve a test accuracy of 94% and an F1-score of 0.95. Furthermore, we achieve an accuracy of 100% on a test set containing normal IMU data, and an F1-score of 0.98 on the test set of abnormal IMU data.

show abstract

Section: Related Workmentioning

confidence: 99%

Anomaly Detection using Deep Reconstruction and Forecasting for Autonomous Systems

Bahavan,

Suman,

Cader

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…A PL model can allow an agent to be able to evaluate abnormalities related to PL and SL models, as it was shown in [3]. However, previous works mostly rely on a high level of supervision to learn PL self-awareness models [13], [15], while in this work, we propose a weakly-supervised method based on a hierarchy of Cross-modal Generative Adversarial Networks (GANs) [16] for estimating PL models. Weakly supervised PL models not can also provide a level of information to boost the SL model as well as they can be used to provide a joint self aware multisensorial modality to cross-predict heterogeneous multimodal anomalies related to the same task execution.…”

Section: Introductionmentioning

confidence: 99%

Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving

Ravanbakhsh

Baydoun

Campo

et al. 2018

2018 21st International Conference on Information Fusion (FUSION)

Self Cite

View full text Add to dashboard Cite

This paper presents a novel approach for learning self-awareness models for autonomous vehicles. Proposed technique is based on the availability of synchronized multi-sensor dynamic data related to different maneuvering tasks performed by a human operator. It is shown that different machine learning approaches can be used to first learn single modality models using coupled Dynamic Bayesian Networks; such models are then correlated at event level to discover contextual multimodal concepts. In the presented case, visual perception and localization are used as modalities. Cross-correlations among modalities in time is discovered from data and are described as probabilistic links connecting shared and private multi-modal DBNs at the event (discrete) level. Results are presented on experiments performed on an autonomous vehicle, highlighting potentiality of the proposed approach to allow anomaly detection and autonomous decision making based on learned self-awareness models.

show abstract

“…Thus, a well-trained model for PL self-awareness can allow an agent to be able to evaluate abnormalities with a more complete information set, based on the joint availability of PL and SL models, as it was shown in [4]. However, previous works mostly rely on a high level of supervision to learn PL self-awareness models [19,7,6,12,20,4], while in this work, we propose a weakly-supervised method based on a hierarchy of Cross-modal Generative Adversarial Networks (GANs) for establishing self-awareness on PL. This model not only can be trained in a self-supervised manner but also can provide a level of information to boost the SL model.…”

Section: Introductionmentioning

confidence: 99%

“…In this sense, the capability of detecting abnormal situations is an important feature included in self-awareness models as it can allow autonomous systems to anticipate in time their situation/contextual awareness about the effectiveness of the decision-making sub-modules [5,6].…”

Section: Introductionmentioning

confidence: 99%

Hierarchy of Gans for Learning Embodied Self-Awareness Model

Ravanbakhsh

Baydoun

Campo

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

In recent years several architectures have been proposed to learn embodied agents complex self-awareness models. In this paper, dynamic incremental self-awareness (SA) models are proposed that allow experiences done by an agent to be modeled in a hierarchical fashion, starting from more simple situations to more structured ones. Each situation is learned from subsets of private agent perception data as a model capable to predict normal behaviors and detect abnormalities. Hierarchical SA models have been already proposed using low dimensional sensorial inputs. In this work, a hierarchical model is introduced by means of a cross-modal Generative Adversarial Networks (GANs) processing high dimensional visual data. Different levels of the GANs are detected in a self-supervised manner using GANs discriminators decision boundaries. Real experiments on semi-autonomous ground vehicles are presented.

show abstract

Dynamic representations for autonomous driving

Cited by 8 publications

References 20 publications

Anomaly Detection using Deep Reconstruction and Forecasting for Autonomous Systems

Anomaly Detection using Deep Reconstruction and Forecasting for Autonomous Systems

Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving

Hierarchy of Gans for Learning Embodied Self-Awareness Model

Contact Info

Product

Resources

About