Linear coordinate-descent message-passing for quadratic optimization

Zhang, Guoqiang; Heusdens, Richard

doi:10.1109/icassp.2012.6288301

Cited by 10 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We point out that the quadratic problem considered in [9] is not a special case of the pairwise separable convex program. The edge-potentials in (2) are required to be convex while in [9] there is no explicit conditions on the edge-potentials.…”

Section: Problem Formulationmentioning

confidence: 99%

“…As a generalization of the LiCD algorithm [9], the GLiCD algorithm incorporates feedback from last iteration in computing new messages. The amount of feedback at each node is controlled by a set of parameters, one for each neighbor.…”

Section: Generalized Linear Coordinate-descent Algorithmmentioning

confidence: 99%

“…In this situation, any element in {x u(t) i , u ∈ N (i)} is a good estimate of the optimal solution x * i . We point out when there is no penalty function involved in L (t) ij (·, ·) (i.e., no feedback), the GLiCD algorithm degenerates to the LiCD algorithm [9]. Thus, the GLiCD algorithm is more flexible than the LiCD algorithm.…”

Section: Message Updating Expressionsmentioning

confidence: 99%

“…Due to the quadratic form of the local functions, the messages of the min-sum algorithm are also in a quadratic form. In [9], the linear coordinate-descent (LiCD) algorithm was proposed for the quadratic optimization, where the messages are in a linear form. Further, the LiCD algorithm was shown to have the same sufficient condition for convergence as that of the min-sum algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Generalized linear coordinate-descent message-passing for convex optimization

Zhang

Heusdens

2012

2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In this paper we propose a generalized linear coordinate-descent (GLiCD) algorithm for a class of unconstrained convex optimization problems. The considered objective function can be decomposed into edge-functions and node-functions of a graphical model. The messages of the GLiCD algorithm are in a form of linear functions, as compared to the min-sum algorithm of which the form of messages depends on the objective function. Thus, the implementation of the GLiCD algorithm is much simpler than that of the min-sum algorithm. A theorem is stated according to which the algorithm converges to the optimal solution if the objective function satisfies a diagonal-dominant condition. As an application, the GLiCD algorithm is exploited in solving the averaging problem in sensor networks, where the performance is compared to that of the min-sum algorithm.

show abstract

Section: Problem Formulationmentioning

confidence: 99%

Section: Generalized Linear Coordinate-descent Algorithmmentioning

confidence: 99%

Section: Message Updating Expressionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generalized linear coordinate-descent message-passing for convex optimization

Zhang

Heusdens

2012

2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are other variants of GaBP algorithm such as Linear Coordinate Descent (LiCD) [201] and Generalized Linear Coordinate Descent (GLiCD) [200] message passing algorithms that are optimized in terms of computation, storage and required transmission bandwidth between the nodes. An example application of GLiCD in distributed beamforming can be found in [77].…”

Section: Belief Propagation On Pgmsmentioning

confidence: 99%

Distributed Processing of Blind Source Separation

Alavi¹

View full text Add to dashboard Cite

<p>Communication is performed by transmitting signals through a medium. It is common that signals originating from different sources are mixed in the transport medium. The operation of separating source signals without prior information about the sources is referred to as blind source separation (BSS). Blind source separation for wireless sensor networks has recently received attention because of low cost and the easy coverage of large areas. Distributed processing is attractive as it is scalable and consumes low power. Existing distributed BSS algorithms either require a fully connected pattern of connectivity, to ensure the good performance, or require a high computational load at each sensor node, to enhance the scalability. This motivates us to develop distributed BSS algorithms that can be implemented over any arbitrary graph with fully shared computations and with good performance. This thesis presents three studies on distributed algorithms. The first two studies are on existing distributed algorithms that are used in linearly constrained convex optimization problems, which are common in signal processing and machine learning. The studies are aimed at improving the algorithms in terms of computational complexity, communication cost, processors coordination and scalability. This makes them more suitable for implementation on sensor networks, thus forming a basis for the development of distributed BSS algorithms on sensor networks in our third study. In the first study, we consider constrained problems in which the constraint includes a weighted sum of all the decision variables. By formulating a constrained dual problem associated to the original constrained problem, we were able to develop a distributed algorithm that can be run both synchronously and asynchronously on any arbitrary graph with lower communication cost than traditional distributed algorithms. In the second study, we consider constrained problems in which the constraint is separable. By making use of the augmented Lagrangian function and splitting the dual variable (Lagrange multiplier) associated to each partial constraint, we were able to develop a distributed fully asynchronous algorithm with lower computational complexity than traditional distributed algorithms. The simplicity of the algorithm is the consequence of approximating the constraint on the equality of the decoupled dual variables. We also provide a measure of the inaccuracy in such an approximation on the optimal value of the primal objective function. Finally, in the third study, we investigate distributed processing solutions for BSS on sensor networks. We propose two distributed processing schemes for BSS that we refer to as scheme 1 and scheme 2. In scheme 1, each sensor node estimates one specific source signal while in scheme 2, by formulating a consensus optimization problem, each sensor node estimates all source signals in a fully shared computation manner. Our proposed algorithms carry the following features: low computational complexity, low power consumption, low data transmission rate, scalability and excellent performance over arbitrary graphs. Although all of our proposed algorithms share the aforementioned properties, each of them is superior in one or some of the features compared to the others. Comparative experimental results show that among all our proposed distributed BSS algorithms, a variant of scheme 1 performs best when all features are considered. This is achieved by making use of the concept of pairwise mutual information along with adding a sparsity assumption on the parameters of the model that is used in BSS.</p>

show abstract

Convergence of Min-Sum-Min Message-Passing for Quadratic Optimization

Zhang

Heusdens

2014

Machine Learning and Knowledge Discovery in Databases

Self Cite

View full text Add to dashboard Cite

We propose a new message-passing algorithm for the quadratic optimization problem. As opposed to the min-sum algorithm, the new algorithm involves two minimizations and one summation at each iteration. The new min-sum-min algorithm exploits feedback from last iteration in generating new messages, resembling the Jacobirelaxation algorithm. We show that if the feedback signal is large enough, the min-sum-min algorithm is guaranteed to converge to the optimal solution. Experimental results show that the min-sum-min algorithm outperforms two reference methods w.r.t. the convergence speed.

show abstract

Linear coordinate-descent message-passing for quadratic optimization

Cited by 10 publications

References 12 publications

Generalized linear coordinate-descent message-passing for convex optimization

Generalized linear coordinate-descent message-passing for convex optimization

Distributed Processing of Blind Source Separation

Convergence of Min-Sum-Min Message-Passing for Quadratic Optimization

Contact Info

Product

Resources

About