Partially Asynchronous, Parallel Algorithms for Network Flow and Other Problems

Abstract-This paper presents a unifying convergence result for asynchronous iterations involving pseudo-contractions in the block-maximum norm. Contrary to previous results which only established asymptotic convergence or studied simplified models of asynchronism, our result allows to bound the convergence rates for both partially and totally asynchronous implementations. Several examples are worked out to demonstrate that our theorem recovers and improves on existing results, and that it allows to characterize the solution times for several classes of asynchronous iterations that have not been addressed before. I. INTRODUCTIONAsynchronous algorithms appear naturally in parallel and distributed systems and are heavily exploited applications ranging from large-scale linear algebra and optimization to distributed coordination of small embedded devices. Allowing nodes to operate in an asynchronous manner simplifies the implementation of distributed algorithms and eliminates the overhead associated with synchronization. However, care has to be taken, since asynchrony runs the risk of rendering an otherwise stable iteration unstable.The dynamics of asynchronous iterations are much richer than their synchronous counterparts, and quantifying the impact of asynchrony on the convergence properties of iterative algorithms remains challenging. Some of the first results on the convergence of asynchronous iterations were derived by Chazan and Miranker [1], who studied chaotic relaxations for solving linear systems of equations. Several authors have proposed extensions of this pioneering work to nonlinear iterations involving maximum norm contractions (e.g., [2], [3]) and for monotone iterations (e.g., [4], [5]). Powerful convergence results for broad classes of asynchronous algorithms, including maximum norm contractions and monotone mappings, under different assumptions on communication delays and update rates were presented by Bertsekas [6] and Bertsekas and Tsitsiklis [7]. Most of the results in the literature only guarantee asymptotic convergence. This paper complements the existing work by developing convergence theorems that characterize the rate of convergence of asynchronous iterations and quantify how these rates depend on the update intervals and information delays in the system.We focus on iterations involving block-maximum norm pseudo-contractions under the general asynchronous model introduced in [6], [7], which allows for heterogeneous and time-varying update rates and communication delays. Such iterations arise in a variety of algorithms, such as certain classes of linear fixed-point iterations and gradient

show abstract

“…The second inequality follows from (18). Therefore, according to Theorem 1, the sequence {x(t)} generated by (2) satisfies (16).…”

Section: Resultsmentioning

confidence: 88%

“…There are several open issues for future work, such as attempting to derive convergence rates of asynchronous iterations involving monotone mappings [16], pseudocontractions with respect to the Euclidean norm [17], and non-expansive mappings [18], much as was done in [19] for the case of non-expansive linear iterations with delays.…”

Section: Discussionmentioning

confidence: 99%

On the convergence rates of asynchronous iterations

Feyzmahdavian

Johansson

2014

53rd IEEE Conference on Decision and Control

View full text Add to dashboard Cite

show abstract

“…We will discuss choices of the space decomposition (4) and the corresponding estimates for C 1 , C 2 , c in (5), (6), (7). In the case of nonlinear network flow, we will also relate our asynchronous method to those studied in [5, §7.2.3], [56]. n j=1 is strongly monotone and Lipschitz continuous on K, and we choose a space decomposition (4) such that each K i is a polyhedral set.…”

Section: Applications To Convex Programmingmentioning

confidence: 99%

“…This estimate further simplifies if A has full row rank, in which case B I is square and invertible. If A does not have full row rank, we could remove the redundant rows, but our experience with network flow problems suggests that this removal can slow the convergence of Gauss-Seidel methods on the problem [56].…”

Section: Dual Applications Consider the Linearly Constrained Convex mentioning

confidence: 99%

“…Our interest in these methods stems from their close connection to relaxation methods for nonlinear network flow (see [4,5,56] and references therein) and to domain decomposition (DD) and multigrid (MG) methods for solving elliptic partial differential equations (see [7,8,9,14,18,19,33,40,45,52,53,57] and references therein). For example, the additive and the multiplicative Schwarz methods may be viewed as Jacobi and Gauss-Seidel type methods applied to linear elliptic partial differential equations reformulated as (1) [9,57].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence rate analysis of an asynchronous space decomposition method for convex Minimization

Tai

Tseng

2001

Math. Comp.

View full text Add to dashboard Cite

Abstract. We analyze the convergence rate of an asynchronous space decomposition method for constrained convex minimization in a reflexive Banach space. This method includes as special cases parallel domain decomposition methods and multigrid methods for solving elliptic partial differential equations. In particular, the method generalizes the additive Schwarz domain decomposition methods to allow for asynchronous updates. It also generalizes the BPX multigrid method to allow for use as solvers instead of as preconditioners, possibly with asynchronous updates, and is applicable to nonlinear problems. Applications to an overlapping domain decomposition for obstacle problems are also studied. The method of this work is also closely related to relaxation methods for nonlinear network flow. Accordingly, we specialize our convergence rate results to the above methods. The asynchronous method is implementable in a multiprocessor system, allowing for communication and computation delays among the processors.

show abstract