On Epistemics in Expected Free Energy for Linear Gaussian State Space Models

Abstract

Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.

Active Data Selection and Information Seeking

“…A reasonable question to ask at this stage is why bother with the full informationseeking objective? It is clear from Figures 4 and 5 that all we need to do is choose to sample from the regions with greatest predicted variance, sequentially reducing that variance until it is minimal throughout the domain of the function being approximated [32]. This suggests that the predictive entropy from Equation ( 2) is sufficient on its own (c.f., maximum entropy sampling [33]).…”

A Bayesian Surprise Approach in Designing Cognitive Radar for Autonomous Driving

“…Compared to the expectation of Bayesian surprise, Equation (A2) captures the filter's uncertainty in interpreting the radar measurements. A recent study adopts the expectation of free energy as a means to investigate exploratory behavior in linear Gaussian dynamic systems [39]. An equivalent analysis of the measurement-selection procedure based on the expectation of Bayesian surprise also applies to the expectation of free energy.…”