a Multi-Perspective Approach to Anomaly Detection for Self -Aware Embodied Agents

Abstract

This paper focuses on multi-sensor anomaly detection for moving cognitive agents using both external and private first-person visual observations. Both observation types are used to characterize agents motion in a given environment. The proposed method generates locally uniform motion models by dividing a Gaussian process that approximates agents displacements on the scene and provides a Shared Level (SL) self-awareness based on Environment Centered (EC) models. Such models are then used to train in a semi-unsupervised way a set of Generative Adversarial Networks (GANs) that produce an estimation of external and internal parameters of moving agents. Obtained results exemplify the feasibility of using multi-perspective data for predicting and analyzing trajectory information.

Incremental Learning of Abnormalities in Autonomous Systems

“…In a lot of existing approaches, this is done offline by trying to take into account a large amount of possible situations. These systems have impressive results in particular environments and situations; nonetheless, they can be totally disoriented in other situations [3,4,10]. Machine learning is one of the promising research areas that try to solve this kind of problems.…”

Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving

“…Recent developments in signal processing and machine learning techniques can be useful to design autonomous systems equipped with a self-awareness module to make it possible to compare how much the current realization is similar to a previous experience of the same type while a given task is executed. The capability of predicting task evolution in normal conditions (i.e., when the task follows the rules learned in the previous experience) and jointly detecting abnormal situations that can rise based on such self-awareness is an important task that allows autonomous systems to increase their situational awareness and the effectiveness of the decision making submodules [2], [3]. Models of different self-awareness layers can be integrated in order to buildup a structured and multi-modal self-aware behavior for an agent.…”

“…In [3] a self-awareness model was introduced that consists of two layers: Shared Level (SL) and Private Layer (PL). The analysis of observed moving agents for learning the models of normal/abnormal dynamics in a given scene from an external viewpoint, represents an emerging research field [4][5][6][7][8][9][10][11][12].…”

“…An external observer has no access to such information, so not being able to directly detect PL abnormalities, while it can still evaluate SL abnormalities using third person models including shared variables. A PL model can allow an agent to be able to evaluate abnormalities related to PL and SL models, as it was shown in [3]. However, previous works mostly rely on a high level of supervision to learn PL self-awareness models [13], [15], while in this work, we propose a weakly-supervised method based on a hierarchy of Cross-modal Generative Adversarial Networks (GANs) [16] for estimating PL models.…”

Hierarchy of Gans for Learning Embodied Self-Awareness Model

“…Self-awareness models make it possible for an agent to evaluate whether faced situations at a given time correspond to previous experiences. Self-aware computational models have been studied and several architectures have been introduced [1,2,3,4]. Such models have to provide a framework where autonomous decisions and/or teleoperation by a human can be integrated as a capability of the device itself to dynamically evaluate the contextual situation [3].…”