Active Inference Integrated With Imitation Learning for Autonomous Driving

Abstract

Classical imitation learning methods suffer substantially from the learning hierarchical policies when the imitative agent faces an unobserved state by the expert agent. To address these drawbacks, we propose an online active learning through active inference approach that encodes the expert's demonstrations based on observation-action to improve the learner's future motion prediction. For this purpose, we provide a switching Dynamic Bayesian Network based on the dynamic interaction between the expert agent and another object in its surrounding as a reference model, which we exploit to initialize an incremental probabilistic learning model. This learning model grows and matures based on the freeenergy formulation and message passing of active inference dynamically at discrete and continuous levels in an online active learning phase. In this scheme, generalized states of the learning world are represented as distance-vector, where it is the learner's observation concerning its interaction with a moving object. Considering the distance vector entail intentions, it enables action prediction evaluation in a prospective sense. We illustrate these points using simulations of driving intelligent agents. The learning agent is trained by using long-term predictions from the generative learning model to reproduce the expert's motion while learning how to select a suitable action through new experiences. Our results affirm that a Dynamic Bayesian optimal approach provides a principled framework and outperforms conventional reinforcement learning methods. Furthermore, it endorses the general formulation of action prediction as active inference.

Exploring Action-Oriented Models via Active Inference for Autonomous Vehicles

“…Then, the learning process is presented, consisting of offline and online phases. Finally, the action-oriented model and the learning cost are discussed and the performance of the proposed approach is compared with two benchmark schemes, AIL [16] and Q-learning [41].…”

“…In this paper, motivated by the above discussion and previous work [16], we introduce a self-awareness framework empowered by Active Inference to improve Autonomous Driving (AD). The proposed framework consists of three main modules: a multi-modal perception module, a global learning module (world model) and an active learning module.…”

Incremental Learning of Goal-Directed Actions in a Dynamic Environment by a Robot Using Active Inference

“…Other studies have also considered the use of incremental learning in teaching robots in a human-robot collaborative setting, such as in [57], where the authors used Gaussian processes (GP) to determine the model's task uncertainty in a manner similar to our use of free energy, although GPs were not used for model optimization. An intriguing proposed method [58] takes an active inference approach to reinforcement learning (RL) by imitation, using free energy instead of typical reward functions. Our proposed scheme utilizes human tutoring combined with mental rehearsal [14,16] for incremental learning, and integrates them with our active inference-based model that employs free-energy minimization in the latent space to find an optimal posterior predictive distribution.…”

How Do Autonomous Vehicles Decide?

“…By reviewing the literature carefully, we highlight some of the main limitations of IL approaches (i) data-hungry deep learning technique and its performance is limited to the level of the expert policy; (ii) expensive or even impossible to obtain supervised data in some circumstances; (iii) due to the real-time nature of many applications, the learning algorithms are constrained by computing power and memory limitations, particularly in robotic applications that require onboard computation to perform the real-time processing [ 99 ]; (iv) traditional IL methods suffer significantly from learning hierarchical policies when the imitative agent encounters an unobserved state by the expert agent [ 100 ]; (v) the policy never outperforms the suboptimal expert performance, and the effectiveness of IL is still heavily dependent on the expert policy’s quality. Furthermore, for a ready reference, we also direct the readers to some recent surveys [ 101 , 102 ] to deep dive into these approaches.…”