Scalable Distributed Belief Propagation with Prioritized Block Updates

Abstract

Belief propagation (BP) is a popular method for performing approximate inference on probabilistic graphical models. However, its message updates are time-consuming, and the schedule for updating messages is crucial to its running time and even convergence. In this paper, we propose a new scheduling scheme that selects a set of messages to update at a time and leverages a novel priority to determine which messages are selected. Additionally, an incremental update approach is introduced to accelerate the computation of the priority. As the size of the model grows, it is desirable to leverage the parallelism of a cluster of machines to reduce the inference time. Therefore, we design a distributed framework, Prom, to facilitate the implementation of BP algorithms. We evaluate the proposed scheduling scheme (supported by Prom) via extensive experiments on a local cluster as well as the Amazon EC2 cloud. The evaluation results show that our scheduling scheme outperforms the state-of-the-art counterpart.

Block Belief Propagation for Parameter Learning in Markov Random Fields

“…Stochastic BP (Noorshams and Wainwright 2013) updates only one dimension of the messages at each inference iteration, so its iteration complexity is much lower than traditional BP. Distributed BP (Schwing et al 2011;Yin and Gao 2014) distributes and parallelizes the computation of beliefs and messages on a cluster of machines to reduce inference time. Sparse-matrix BP (Bixler and Huang 2018) uses sparsematrix products to represent the message-passing indexing, so that it can be implemented on modern hardware.…”