Fixing convergence of Gaussian belief propagation

Abstract

Abstract-Gaussian belief propagation (GaBP) is an iterative message-passing algorithm for inference in Gaussian graphical models. It is known that when GaBP converges it converges to the correct MAP estimate of the Gaussian random vector and simple sufficient conditions for its convergence have been established.In this paper we develop a double-loop algorithm for forcing convergence of GaBP. Our method computes the correct MAP estimate even in cases where standard GaBP would not have converged. We further extend this construction to compute least-squares solutions of over-constrained linear systems. We believe that our construction has numerous applications, since the GaBP algorithm is linked to solution of linear systems of equations, which is a fundamental problem in computer science and engineering. As a case study, we discuss the linear detection problem. We show that using our new construction, we are able to force convergence of Montanari's linear detection algorithm, in cases where it would originally fail. As a consequence, we are able to increase significantly the number of users that can transmit concurrently.

Linear coordinate-descent message-passing for quadratic optimization

“…Denote Jα = J + αI. It was shown in [7] that by following the iteration (15),x (t) converges to the solution (14) for any initializationx (0) . Givenx (t) at time t, the new estimatex (t+1) is computed by exploiting the min-sum algorithm.…”

“…Recently, the min-sum algorithm has been successfully applied for the problem [12,7]. As an alternative solution, we consider applying the LiCD algorithm in solving the same problem.…”

“…Recent work by Moallemi and Roy provided a geometrical meaning to the walk-summability of J [6]. In [7], the min-sum algorithm for quadratic optimization was applied in solving general linear systems.…”

“…In addition, we consider applying the LiCD algorithm in solving general linear systems. We follow the line of work in [7], where the min-sum algorithm is exploited in solving the same problem. We found by experiment that the LiCD algorithm is comparable to the min-sum algorithm in convergence speed, but has a lower computational complexity and requires less storage capacity.…”

Graffiti: graph-based classification in heterogeneous networks

“…Recently, a double-loop technique was introduced to force convergence of the Jacobi algorithm to the correct solution even when the sufficient conditions for convergence do not hold. The double loop technique works for either positive definite or column dependent matrices [29]. Thus, the PageRank computation converges with the rate of convergence determined by the Jacobi method.…”