Stochastic Belief Propagation: A Low-Complexity Alternative to the Sum-Product Algorithm

Abstract

The sum-product or belief propagation (BP) algorithm is a widely-used messagepassing algorithm for computing marginal distributions in graphical models with discrete variables. At the core of the BP message updates, when applied to a graphical model with pairwise interactions, lies a matrix-vector product with complexity that is quadratic in the state dimension d, and requires transmission of a (d − 1)-dimensional vector of real numbers (messages) to its neighbors. Since various applications involve very large state dimensions, such computation and communication complexities can be prohibitively complex. In this paper, we propose a low-complexity variant of BP, referred to as stochastic belief propagation (SBP). As suggested by the name, it is an adaptively randomized version of the BP message updates in which each node passes randomly chosen information to each of its neighbors. The SBP message updates reduce the computational complexity (per iteration) from quadratic to linear in d, without assuming any particular structure of the potentials, and also reduce the communication complexity significantly, requiring only log d bits transmission per edge. Moreover, we establish a number of theoretical guarantees for the performance of SBP, showing that it converges almost surely to the BP fixed point for any tree-structured graph, and for graphs with cycles satisfying a contractivity condition. In addition, for these graphical models, we provide non-asymptotic upper bounds on the convergence rate, showing that the ℓ ∞ norm of the error vector decays no slower than O 1/ √ t with the number of iterations t on trees and the mean square error decays as O 1/t for general graphs. These analysis show that SBP can provably yield reductions in computational and communication complexities for various classes of graphical models.

User Association With Unequal User Priorities in Heterogeneous Cellular Networks

“…Whether the BP algorithm can converge to a fixed point, unfortunately, is not well understood yet [26]. Generally speaking, if the factor graph is sparse and contains no cycles, the BP algorithm can converge to a fixed point exactly and efficiently [27]. In Section V, we will analyze the average sparsity of our factor graph based on the stochastic geometry theory.…”

“…Thus, we can have closed forms for D u , D l , and S in (27), (28), and (29), respectively, when α = 4.…”

“…By substituting the above Z into (23) and (24), we can obtain (27) and (28) respectively. This completes the proof.…”

A Principled Approach for Coarse-to-Fine MAP Inference

“…For certain pairwise potentials, that are important in computer vision applications, this minconvolution step can be done very quickly [13], [14], [15]. Recent approaches for efficient message passing in large state spaces are stochastic belief propagation [16], which only considers a random subset of states for message computation, and the use of a trained classifier to prune the state space [17].…”

An Energy-Efficient Collaborative Ground Vibration Measurement Scheme Based on Compressed Sensing and Belief Propagation

“…Furthermore, the computation complexity will increase exponentially. BP is a highly efficient message fusion algorithm for performing inference on graphical models in the machine learning field [28]. BP utilizes inference networks to model the correlation of nodes, changes the global integration into local integration, and then passes the message through the networks and obtains the evaluation of the probability distribution.…”