Smoothed Analysis of Belief Propagation for Minimum-Cost Flow and Matching

Abstract

Belief propagation (BP) is a message-passing heuristic for statistical inference in graphical models such as Bayesian networks and Markov random fields. BP is used to compute marginal distributions or maximum likelihood assignments and has applications in many areas, including machine learning, image processing, and computer vision. However, the theoretical understanding of the performance of BP remains limited.Recently, BP has been applied to combinatorial optimization problems. It has been proved that BP can be used to compute maximum-weight matchings and minimum-cost flows for instances with a unique optimum. The number of iterations needed for this is pseudo-polynomial and hence BP is not efficient in general.We study BP in the framework of smoothed analysis and prove that with high probability the number of iterations needed to compute maximum-weight matchings and minimum-cost flows is bounded by a polynomial if the weights/costs of the edges are randomly perturbed. To prove our upper bounds, we use an isolation lemma by Beier and Vöcking (SIAM Journal on Computing, 2006) for the matching problem and we generalize an isolation lemma by Gamarnik, Shah, and Wei (Operations Research, 2012) for the min-cost flow problem. We also prove lower tail bounds for the number of iterations that BP needs to converge that almost match our upper bounds. Belief PropagationThe belief propagation (BP) algorithm is a message-passing algorithm that is used for solving probabilistic inference problems on graphical models. It was proposed by Pearl in 1988 [9]. Typical graphical models to which BP is applied are Bayesian networks and Markov random fields. There are two variants of BP. The sum-product variant is used to compute marginal probabilities. The max-product variant (or the functionally equivalent min-sum variant) is used to compute maximum a posteriori (MAP) probability estimates.

Smoothed Differential Privacy

“…The smoothed analysis [43] is a widely-accepted analysis tool in machine learning [26,34], computational social choice [50,7,51,32], and etc. [12,8,9]. In differential privacy literature, the smoothed analysis is a widely-accepted tool to calculate the sensitivity of mechanisms under realistic setting [39,14].…”

Abstractive Text Summarization: Enhancing Sequence-to-Sequence Models Using Word Sense Disambiguation and Semantic Content Generalization

“…This approach reduces optimal bipartite matching to a minimum cost flow problem (Hansen and Klopfer 2006;Dasgupta, Papadimitriou, and Vazirani 2008;Brunsch et al 2013). More specifically, the minimum cost flow graph G (as, for example, in Figure 4) consists of an source node Src, the set of nodes of the general concepts N G (i.e.…”

Probability Conditions for Convergence of the Warning Propagation Algorithm

“…At the same time, this paper also provides a simplified BP algorithm, which can give a fully polynomial-time randomized approximation scheme (FPRAS) for the MCF problem. Reference [15] gives a conditional probability of the BP algorithm convergence. The results show that BP algorithm can give the optimal solution of the maximum weight matching problem and the minimum cost network flow problem, and the times of algorithm iterations is polynomial bound on high probability.…”