Simulating Active Inference Processes by Message Passing

Laar, T.W. van de; Vries, Bert de

doi:10.3389/frobt.2019.00020

Cited by 32 publications

(38 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One can interpret this as the agent having prior beliefs over states that it will visit, independent of a policy, but driving policy selection to these attractor states. An obvious example of a preferred state is for example maintaining a temperature of 37 °C (Van De Laar and De Vries, 2019 ). These preferred states basically determine the agent, and can be endowed on the agent either by nature through evolution, in the case of natural agents, or by humans in the case of artificial agents.…”

Section: Methodsmentioning

confidence: 99%

“…Current active inference schemes usually start by specifying the belief state space manually, in terms of a generative model. Variational Bayes is then used to infer hidden states and parameters under this model (Friston et al, 2009 ; Sajid et al, 2019 ; Van De Laar and De Vries, 2019 ). This approach works well for low-dimensional problems or problems where a sensible belief state can be devised for the task at hand.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Learning Generative State Space Models for Active Inference

Çatal

Wauthier

Boom

et al. 2020

Front. Comput. Neurosci.

View full text Add to dashboard Cite

In this paper we investigate the active inference framework as a means to enable autonomous behavior in artificial agents. Active inference is a theoretical framework underpinning the way organisms act and observe in the real world. In active inference, agents act in order to minimize their so called free energy, or prediction error. Besides being biologically plausible, active inference has been shown to solve hard exploration problems in various simulated environments. However, these simulations typically require handcrafting a generative model for the agent. Therefore we propose to use recent advances in deep artificial neural networks to learn generative state space models from scratch, using only observation-action sequences. This way we are able to scale active inference to new and challenging problem domains, whilst still building on the theoretical backing of the free energy principle. We validate our approach on the mountain car problem to illustrate that our learnt models can indeed trade-off instrumental value and ambiguity. Furthermore, we show that generative models can also be learnt using high-dimensional pixel observations, both in the OpenAI Gym car racing environment and a real-world robotic navigation task. Finally we show that active inference based policies are an order of magnitude more sample efficient than Deep Q Networks on RL tasks.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Learning Generative State Space Models for Active Inference

Çatal

Wauthier

Boom

et al. 2020

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…In summary, we can interpret the dynamics of a system described by mean-field density dynamics in terms of messages (i.e., mean-fields) passed between module-like regions of a network [ 69 , 70 , 71 ]. For sufficiently sparse conditional dependency structures—like that of the Hamiltonian employed here—the message passing is evocative of synaptic communication in sparse neuronal networks.…”

Section: Neuronal Message Passingmentioning

confidence: 99%

Modules or Mean-Fields?

Parr

Sajid

Friston

2020

Entropy

View full text Add to dashboard Cite

The segregation of neural processing into distinct streams has been interpreted by some as evidence in favour of a modular view of brain function. This implies a set of specialised ‘modules’, each of which performs a specific kind of computation in isolation of other brain systems, before sharing the result of this operation with other modules. In light of a modern understanding of stochastic non-equilibrium systems, like the brain, a simpler and more parsimonious explanation presents itself. Formulating the evolution of a non-equilibrium steady state system in terms of its density dynamics reveals that such systems appear on average to perform a gradient ascent on their steady state density. If this steady state implies a sufficiently sparse conditional independency structure, this endorses a mean-field dynamical formulation. This decomposes the density over all states in a system into the product of marginal probabilities for those states. This factorisation lends the system a modular appearance, in the sense that we can interpret the dynamics of each factor independently. However, the argument here is that it is factorisation, as opposed to modularisation, that gives rise to the functional anatomy of the brain or, indeed, any sentient system. In the following, we briefly overview mean-field theory and its applications to stochastic dynamical systems. We then unpack the consequences of this factorisation through simple numerical simulations and highlight the implications for neuronal message passing and the computational architecture of sentience.

show abstract

“…With these preliminaries in place, we can start to think about the implications this has for the neural architectures that solve particular types of inference problem. Figure 1 shows a simple graphical representation of a generative model and an interpretation of the computations required to find posterior distributions in terms of the passing of local messages [ 11 , 12 , 24 ]. The graphical representation on the right shows the dependencies implicit in the Bayes optimal updating of beliefs.…”

Section: Graphical Models and Inferencementioning

confidence: 99%

“…However, our primary focus here is on characterising priors for policy selection, and we will gloss over the details of these inference schemes and assume exact Bayesian inference is tractable. While there may be subtle differences resulting from the application of different message-passing schemes [ 11 , 24 , 28 ], this will not influence the computational anatomy at the level of description adopted in this paper, which rests upon conditional dependencies or Markov blankets in a generative model. Markov blankets are statistical constructs that partition sets of random variables.…”

Section: Graphical Models and Inferencementioning

confidence: 99%

Inferring What to Do (And What Not to)

Parr

2020

Entropy

View full text Add to dashboard Cite

In recent years, the “planning as inference” paradigm has become central to the study of behaviour. The advance offered by this is the formalisation of motivation as a prior belief about “how I am going to act”. This paper provides an overview of the factors that contribute to this prior. These are rooted in optimal experimental design, information theory, and statistical decision making. We unpack how these factors imply a functional architecture for motivated behaviour. This raises an important question: how can we put this architecture to work in the service of understanding observed neurobiological structure? To answer this question, we draw from established techniques in experimental studies of behaviour. Typically, these examine the influence of perturbations of the nervous system—which include pathological insults or optogenetic manipulations—to see their influence on behaviour. Here, we argue that the message passing that emerges from inferring what to do can be similarly perturbed. If a given perturbation elicits the same behaviours as a focal brain lesion, this provides a functional interpretation of empirical findings and an anatomical grounding for theoretical results. We highlight examples of this approach that influence different sorts of goal-directed behaviour, active learning, and decision making. Finally, we summarise their implications for the neuroanatomy of inferring what to do (and what not to).

show abstract

Simulating Active Inference Processes by Message Passing

Cited by 32 publications

References 42 publications

Learning Generative State Space Models for Active Inference

Learning Generative State Space Models for Active Inference

Modules or Mean-Fields?

Inferring What to Do (And What Not to)

Contact Info

Product

Resources

About