Message-Passing Algorithms: Reparameterizations and Splittings

Abstract

The max-product algorithm, a local message-passing scheme that attempts to compute the most probable assignment (MAP) of a given probability distribution, has been successfully employed as a method of approximate inference for applications arising in coding theory, computer vision, and machine learning. However, the max-product algorithm is not guaranteed to converge to the MAP assignment, and if it does, is not guaranteed to recover the MAP assignment. Alternative convergent message-passing schemes have been proposed to overcome these difficulties. This work provides a systematic study of such message-passing algorithms that extends the known results by exhibiting new sufficient conditions for convergence to local and/or global optima, providing a combinatorial characterization of these optima based on graph covers, and describing a new convergent and correct message-passing algorithm whose derivation unifies many of the known convergent message-passing algorithms. While convergent and correct message-passing algorithms represent a step forward in the analysis of max-product style message-passing algorithms, the conditions needed to guarantee convergence to a global optimum can be too restrictive in both theory and practice. This limitation of convergent and correct message-passing schemes is characterized by graph covers and illustrated by example.Comment: A complete rework and expansion of the previous version

Receiver optimization on non‐binary joint sparse graph for OFDM system

“…First, we analyze the iterative structure of the non-binary joint sparse graph. As shown in (19) and (20), variable nodes calculate their extrinsic messages using a priori information which they receive from other connected chip nodes and parity check nodes. By contrast, the updating of chip nodes and parity check nodes both only depend one type of a priori information coming from variable nodes.…”

“…Based on Figure 3 (a), we creatively introduce a special type of node, the syndrome node syn k , into the iterative structure of the non-binary joint sparse graph. The optimized structure is schematically shown in Figure 3 (b), where the syndrome nodes (diamonds) are drawn on the More explicitly, when the message L vk;m→cn is updated (see (19)), we first take an extra processing on the message L p k;j →vk;m . For instance, in a typical iteration, if the L p k;j →vk;m comes from the parity check node whose syndrome value equals to zero, it is reasonable to judge that such L p k;j →vk;m is highly dependable, and we multiply the L p k;j →vk;m with a coefficient δ, where δ ≥ 1 and can be adapted to the channel condition.…”

“…Otherwise, L p k;j →vk;m is processed in the conventional manner. Hence, the expression of (19) where syn j = 0 and syn j ≠ 0 represent the parity check nodes whose syndromes whether equal to zeros. Similarly, the updating of L vk;m→p k;j , which is shown in (20), also can be modified according to the optimized structure in Figure 3 (b), that is, To compare the two different structures of Figure 3, syndrome distributions of the same user investigated in Figure 4 are recalculated on the optimized structure and shown in Figure 5.…”

“…In information theory, the mutual information is a measure of the variables' mutual dependence. For the non‐binary joint sparse graph, convergence rate is determined by the intersection point and the trajectory to such intersection point of EXIT charts , which, respectively, means the convergence point of joint MUD and channel decoding and the number of iterations of the processing. By adding syndrome nodes, above both two factors are optimized; thus, the convergence rate of the non‐binary joint sparse graph is improved.…”

Analysis of the Min-Sum Algorithm for Packing and Covering Problems via Linear Programming

“…Bayati et al [BBCZ11] write that their "proof gives a better understanding of the often-noted but poorly understood connection between BP and LP through graph covers." We further clarify this connection by using higher order covers that capture fractional optimal LP solutions, as suggested by Ruozzi and Tatikonda [RT12]. Graph covers not only capture LP solutions but also solutions computed by the min-sum algorithm.…”