Task Parallel Implementation of Belief Propagation in Factor Graphs

Ma, Ning; Xia, Yinglong; Prasanna, Viktor K.

doi:10.1109/ipdpsw.2012.238

Cited by 3 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since both belief propagation and learning are totally local, they can be implemented with distributed hardware or parallelized processes. Some studies have been carried out for other deep network frameworks (Liang et al, 2009), (Silberstein et al, 2008), (Ma et al, 2012)) and we are confident that similarly the FGrn paradigm may present new interesting opportunities to approach some of the most challenging tasks in computer vision.…”

Section: Discussionmentioning

confidence: 79%

Towards Building Deep Networks with Bayesian Factor Graphs

Buonanno¹,

Palmieri²

2015

Preprint

View full text Add to dashboard Cite

We propose a Multi-Layer Network based on the Bayesian framework of the Factor Graphs in Reduced Normal Form (FGrn) applied to a two-dimensional lattice. The Latent Variable Model (LVM) is the basic building block of a quadtree hierarchy built on top of a bottom layer of random variables that represent pixels of an image, a feature map, or more generally a collection of spatially distributed discrete variables. The multi-layer architecture implements a hierarchical data representation that, via belief propagation, can be used for learning and inference. Typical uses are pattern completion, correction and classification. The FGrn paradigm provides great flexibility and modularity and appears as a promising candidate for building deep networks: the system can be easily extended by introducing new and different (in cardinality and in type) variables. Prior knowledge, or supervised information, can be introduced at different scales. The FGrn paradigm provides a handy way for building all kinds of architectures by interconnecting only three types of units: Single Input Single Output (SISO) blocks, Sources and Replicators. The network is designed like a circuit diagram and the belief messages flow bidirectionally in the whole system. The learning algorithms operate only locally within each block. The framework is demonstrated in this paper in a three-layer structure applied to images extracted from a standard data set.

show abstract

Section: Discussionmentioning

confidence: 79%

Towards Building Deep Networks with Bayesian Factor Graphs

Buonanno¹,

Palmieri²

2015

Preprint

View full text Add to dashboard Cite

show abstract

“…Distributed implementation, associated ease of programming and strong parallelization potential are the main reasons for the growing popularity of the BP algorithm. For example, several software architectures for implementing parallel BPs were recently proposed [15,11,16].…”

Section: Introductionmentioning

confidence: 99%

Convergence and Correctness of Max-Product Belief Propagation for Linear Programming

Park¹,

Shin²

2017

SIAM J. Discrete Math.

View full text Add to dashboard Cite

The max-product belief propagation (BP) is a popular message-passing heuristic for approximating a maximum-a-posteriori (MAP) assignment in a joint distribution represented by a graphical model (GM). In the past years, it has been shown that BP can solve a few classes of linear programming (LP) formulations to combinatorial optimization problems including maximum weight matching, shortest path and network flow, i.e., BP can be used as a message-passing solver for certain combinatorial optimizations. However, those LPs and corresponding BP analysis are very sensitive to underlying problem setups, and it has been not clear what extent these results can be generalized to. In this paper, we obtain a generic criteria that BP converges to the optimal solution of given LP, and show that it is satisfied in LP formulations associated to many classical combinatorial optimization problems including maximum weight perfect matching, shortest path, traveling salesman, cycle packing, vertex/edge cover and network flow. methods [14] for large-scale inputs. However, these theoretical results on BP are very sensitive to underlying structural properties depending on specific problems and it is not clear what extent they can be generalized to, e.g., the BP analysis for matching problems [3,21,12,20] does not extend to even for perfect matching ones [2]. In this paper, we overcome such technical difficulties for enhancing the power of BP as a LP solver. ContributionWe establish a generic criteria for GM formulations of given LP so that BP converges to the optimal LP solution given arbitrary initialization. Consequently, it also provides a sufficient condition for guaranteeing that a BP fixed point is unique. As one can naturally expect given prior results, one of our conditions requires the LP tightness. Our main contribution is finding other sufficient generic conditions so that BP converges to the correct MAP assignment of GM. First of all, our generic criteria can rediscover all prior BP results on this line, including matching [3, 21, 12], perfect matching [2], matching with odd cycles [24] and shortest path [19], i.e., we provide a unified framework on establishing the convergence and correctness of BPs in relation to associated LPs. Furthermore, we provide new instances under our framework: we show that BP can solve LP formulations associated to other popular combinatorial optimizations including perfect matching with odd cycles, traveling salesman, cycle packing, network flow and vertex/edge cover, which are not known in the literature. Here, we remark that the same network flow problem was already studied using BP by Gamarnik et al. [10]. However, our BP is different from theirs and much simpler to implement/analyze: the authors study BP on continuous GMs, and we do BP on discrete GMs. While most prior known BP results on this line focused on the case when the associated LP has an integral solution, the proposed criteria naturally guides the BP design to compute fractional LP solutions as well (see Section 4.2 and Section 4.4...

show abstract