Active Inference for Integrated State-Estimation, Control, and Learning

Abstract

This work presents an approach for control, stateestimation and learning model (hyper)parameters for robotic manipulators. It is based on the active inference framework, prominent in computational neuroscience as a theory of the brain, where behaviour arises from minimizing variational freeenergy. The robotic manipulator shows adaptive and robust behaviour compared to state-of-the-art methods. Additionally, we show the exact relationship to classic methods such as PID control. Finally, we show that by learning a temporal parameter and model variances, our approach can deal with unmodelled dynamics, damps oscillations, and is robust against disturbances and poor initial parameters. The approach is validated on the 'Franka Emika Panda' 7 DoF manipulator.

Active Inference in Robotics and Artificial Agents: Survey and Challenges

“…General introduction and tutorial [50], [5], Derivations for robot control [13], [17] Relationship with classical control [51], [18] Relationship with optimal control [52], [53] Discrete Active Inference [54] RL and active inference [55], [25] State estimation [56], [9] Predictive processing [57], [58] Human Robot Interaction [45] Neuroscientific foundations [1], [59] Fig. 2.…”

“…2) With precision learning: The authors in [18] presented an evolution of [17] by including online parameters learning for controller auto-tuning. The authors demonstrated how the minimization of the VFE produces effective state-estimation, control for robotic manipulators.…”

“…They introduced a temporal parameter τ in the dynamics generative model used in [17] corresponding to a variable time constant of the first-order linear system used as the model reference for the joint motions. Basically, the authors showed that the gains of the controller correspond to the covariance matrices of the observation model, and that learning the optimal covariance matrices results in finding the optimal gains for the controller [18]. The results showed improved performance in terms of response and robustness compared to a manually tuned AIC, but they also showed some limitations of the standard active inference formulation when (hyper-)parameters learning is introduced.…”

“…If one considers the generalized motions up to a third order, the control law would include a non-zero D term. A detailed analysis of how PID arises in active inference under approximate linear generative models can be found in [51], [18].…”

“…[5]). However, this is very challenging for complex systems-see the closed form equation in [63, appendix] when the system is known-and hence it is usually approximated with a directional constant [13], [18], [23] or by performing the control on the proprioceptive states [13], [20]. Potentially interesting alternative solutions exist in the concepts of adaptive interaction [118] or direct gradient descent control [119].…”