Variational Message Passing and Local Constraint Manipulation in Factor Graphs

Senoz, Ismail; Laar, Thijs van de; Bagaev, Dmitry; Vries, Bert de

doi:10.3390/e23070807

Cited by 19 publications

(44 citation statements)

References 46 publications

(103 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We present the ReactiveMP.jl package, which is a native Julia [1] package for automated reactive message passing-based Bayesian inference and corresponding Constrained Bethe Free Energy (CFBE) functional optimisation [6]. ReactiveMP.jl is based on a reactive programming approach, does not enforce any particular message-passing schedule, and supports real-time data inference.…”

Section: Reactive Message Passingmentioning

confidence: 99%

ReactiveMP.jl: A Julia Package for Reactive Message Passing-based Bayesian Inference

2022

JCON

View full text Add to dashboard Cite

ReactiveMP.jl is a native Julia implementation of reactive message passing-based Bayesian inference in probabilistic graphical models with Factor Graphs. The package does Constrained Bethe Free Energy minimisation and supports both exact and variational Bayesian inference, provides a convenient syntax for model specification and allows for extra factorisation and form constraints specification of the variational family of distributions. In addition, ReactiveMP.jl includes a large range of standard probabilistic models and can easily be extended to custom novel nodes and message update rules. In contrast to non-reactive (imperatively coded) Bayesian inference packages, ReactiveMP.jl scales easily to support inference on a standard laptop for large conjugate models with tens of thousands of variables and millions of nodes.

show abstract

Section: Reactive Message Passingmentioning

confidence: 99%

ReactiveMP.jl: A Julia Package for Reactive Message Passing-based Bayesian Inference

2022

JCON

View full text Add to dashboard Cite

show abstract

“…Furthermore, ReactiveMP automatically evaluates the performance of the model on the data by calculating the Bethe free energy [46], which equals the variational free energy for acyclic graphs. This free energy is calculated using node-local free energies and the edge-local entropies of the random variables, where we can also impose local constraints on these variables for a trade-off between tractability and accuracy of the free energy calculation [47].…”

Section: Automating Inference and Variational Free Energy Evaluationmentioning

confidence: 99%

A Bayesian Modeling Approach to Situated Design of Personalized Soundscaping Algorithms

Erp

Podusenko

Ignatenko³

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

Effective noise reduction and speech enhancement algorithms have great potential to enhance lives of hearing aid users by restoring speech intelligibility. An open problem in today’s commercial hearing aids is how to take into account users’ preferences, indicating which acoustic sources should be suppressed or enhanced, since they are not only user-specific but also depend on many situational factors. In this paper, we develop a fully probabilistic approach to “situated soundscaping”, which aims at enabling users to make on-the-spot (“situated”) decisions about the enhancement or suppression of individual acoustic sources. The approach rests on a compact generative probabilistic model for acoustic signals. In this framework, all signal processing tasks (source modeling, source separation and soundscaping) are framed as automatable probabilistic inference tasks. These tasks can be efficiently executed using message passing-based inference on factor graphs. Since all signal processing tasks are automatable, the approach supports fast future model design cycles in an effort to reach commercializable performance levels. The presented results show promising performance in terms of SNR, PESQ and STOI improvements in a situated setting.

show abstract

“…Minimizing F[q] then constitutes an upper bound on − log p(x t |x 1:t−1 ), meaning we can optimise (26) to obtain an approximate solution to our original inference problem. We define the optimal recognition density q * as the one that minimises F[q]:…”

Section: Computing G-expected Free Energymentioning

confidence: 99%

“…For further background on variational inference we refer interested readers to the seminal works by [26,27]. Now we are ready to introduce the EFE.…”

Section: Computing G-expected Free Energymentioning

confidence: 99%

“…Minimizing

then constitutes an upper bound on

, meaning we can optimise ( 26 ) to obtain an approximate solution to our original inference problem. We define the optimal recognition density

as the one that minimises

For further background on variational inference we refer interested readers to the seminal works by [ 26 , 27 ]. Now we are ready to introduce the EFE.…”

Section: Action Selection Under Active Inferencementioning

confidence: 99%

See 1 more Smart Citation

On Epistemics in Expected Free Energy for Linear Gaussian State Space Models

Koudahl

Kouw

Vries

2021

Entropy

View full text Add to dashboard Cite

Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.

show abstract

Variational Message Passing and Local Constraint Manipulation in Factor Graphs

Cited by 19 publications

References 46 publications

ReactiveMP.jl: A Julia Package for Reactive Message Passing-based Bayesian Inference

ReactiveMP.jl: A Julia Package for Reactive Message Passing-based Bayesian Inference

A Bayesian Modeling Approach to Situated Design of Personalized Soundscaping Algorithms

On Epistemics in Expected Free Energy for Linear Gaussian State Space Models

Contact Info

Product

Resources

About