Online Variational Message Passing in the Hierarchical Gaussian Filter

Senoz, Ismail; Vries, Bert de

doi:10.1109/mlsp.2018.8517019

Cited by 11 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since exact inference is not possible, these studies have relied on different approximate inference approaches, such as sampling, Taylor approximation, variational inference or message-passing algorithms. Our approach for treatment of binary observations using moment matching is similar to the message-passing approach taken recently for studying dynamical systems [46,47]. We chose this approach rather than the Taylor approximation used previously [18] because it has been shown that methods based on moment matching perform better than derivative-based methods in approximating exact inference for binary Gaussian process models [48].…”

Section: Discussionmentioning

confidence: 99%

A simple model for learning in volatile environments

Piray

Daw

2020

PLoS Comput Biol

View full text Add to dashboard Cite

Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.

show abstract

Section: Discussionmentioning

confidence: 99%

A simple model for learning in volatile environments

Piray

Daw

2020

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…This aproach scales in principle to more complex applications that may be of interest to industry as well. For example, the state space models in the examples can be readily extended to hierarchical generative models (Kiebel et al, 2009 ; Senoz and de Vries, 2018 ), which have been shown to be quite powerful in modeling real-world dynamics (e.g., Turner and Sahani, 2008 ; Mathys et al, 2014 ).…”

Section: Discussionmentioning

confidence: 99%

Simulating Active Inference Processes by Message Passing

Laar

Vries

2019

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

The free energy principle (FEP) offers a variational calculus-based description for how biological agents persevere through interactions with their environment. Active inference (AI) is a corollary of the FEP, which states that biological agents act to fulfill prior beliefs about preferred future observations (target priors). Purposeful behavior then results from variational free energy minimization with respect to a generative model of the environment with included target priors. However, manual derivations for free energy minimizing algorithms on custom dynamic models can become tedious and error-prone. While probabilistic programming (PP) techniques enable automatic derivation of inference algorithms on free-form models, full automation of AI requires specialized tools for inference on dynamic models, together with the description of an experimental protocol that governs the interaction between the agent and its simulated environment. The contributions of the present paper are two-fold. Firstly, we illustrate how AI can be automated with the use of ForneyLab, a recent PP toolbox that specializes in variational inference on flexibly definable dynamic models. More specifically, we describe AI agents in a dynamic environment as probabilistic state space models (SSM) and perform inference for perception and control in these agents by message passing on a factor graph representation of the SSM. Secondly, we propose a formal experimental protocol for simulated AI. We exemplify how this protocol leads to goal-directed behavior for flexibly definable AI agents in two classical RL examples, namely the Bayesian thermostat and the mountain car parking problems.

show abstract

“…As in the previous example, the exact Bayesian inference for this type of model is not tractable. Moreover, the variational message update rules (9) are not tractable either, as the HGF model contains nonconjugate relationships among variables in the form of the link function f. However, inference in this model is still possible with the custom message update rules approximation [43]. Te ReactiveMP.jl package supports a straightforward API to add custom nodes with custom message passing update rules.…”

Section: Hierarchical Gaussianmentioning

confidence: 99%

Reactive Message Passing for Scalable Bayesian Inference

Bagaev

Vries

2023

Scientific Programming

View full text Add to dashboard Cite

We introduce reactive message passing (RMP) as a framework for executing schedule-free, scalable, and, potentially, more robust message passing-based inference in a factor graph representation of a probabilistic model. RMP is based on the reactive programming style, which only describes how nodes in a factor graph react to changes in connected nodes. We recognize reactive programming as the suitable programming abstraction for message passing-based methods that improve robustness, scalability, and execution time of the inference procedure and are useful for all future implementations of message passing methods. We also present our own implementation ReactiveMP.jl, which is a Julia package for realizing RMP through minimization of a constrained Bethe free energy. By user-defined specification of local form and factorization constraints on the variational posterior distribution, ReactiveMP.jl executes hybrid message passing algorithms including belief propagation, variational message passing, expectation propagation, and expectation maximization update rules. Experimental results demonstrate the great performance of our RMP implementation compared to other Julia packages for Bayesian inference across a range of probabilistic models. In particular, we show that the RMP framework is capable of performing Bayesian inference for large-scale probabilistic state-space models with hundreds of thousands of random variables on a standard laptop computer.

show abstract

Online Variational Message Passing in the Hierarchical Gaussian Filter

Cited by 11 publications

References 6 publications

A simple model for learning in volatile environments

A simple model for learning in volatile environments

Simulating Active Inference Processes by Message Passing

Reactive Message Passing for Scalable Bayesian Inference

Contact Info

Product

Resources

About