New Douglas--Rachford Algorithmic Structures and Their Convergence Analyses

Censor, Yair; Mansour, Rafiq

doi:10.1137/141001536

Cited by 13 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [20], it was shown that many convex minimization and monotone inclusion problems reduce to the more general problem of finding a fixed point of compositions of averaged operators, which provided a unified analysis of various proximal splitting algorithms. Along these lines, several fixed point methods based on various combinations of averaged operators have since been devised, see [1,2,5,9,11,13,14,17,18,24,25,38,46] for recent work. Motivated by deep neural network structures with thus far elusive asymptotic properties, we investigate in the present paper a novel averaged operator model involving a mix of nonlinear and linear operators.…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Network Structures Solving Variational Inequalities

Combettes

Pesquet

2020

Set-Valued Var. Anal

104

View full text Add to dashboard Cite

Motivated by structures that appear in deep neural networks, we investigate nonlinear composite models alternating proximity and affine operators defined on different spaces. We first show that a wide range of activation operators used in neural networks are actually proximity operators. We then establish conditions for the averagedness of the proposed composite constructs and investigate their asymptotic properties. It is shown that the limit of the resulting process solves a variational inequality which, in general, does not derive from a minimization problem.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Neural Network Structures Solving Variational Inequalities

Combettes

Pesquet

2020

Set-Valued Var. Anal

104

View full text Add to dashboard Cite

show abstract

“…String-averaging and block-iterative variants. In 2016, Censor and Mansour introduced the string-averaging DR (SA-DR) and block-iterative DR (BI-DR) variants [60]. SA-DR involves separating the index set I := {1, .…”

Section: Connection With Methods Of Multipliers (Admm)mentioning

confidence: 99%

Survey: Sixty Years of Douglas–rachford

Lindstrom¹,

Sims²

2020

J. Aust. Math. Soc.

View full text Add to dashboard Cite

DedicationThis work is dedicated to the memory of Jonathan M. Borwein our greatly missed friend, mentor, and colleague. His influence on both the topic at hand, as well as his impact on the present authors personally, cannot be overstated. AbstractThe Douglas-Rachford method is a splitting method frequently employed for finding zeroes of sums of maximally monotone operators. When the operators in question are normal cones operators, the iterated process may be used to solve feasibility problems of the form: Find x ∈ N k=1 S k . The success of the method in the context of closed, convex, nonempty sets S 1 , . . . , S N is well-known and understood from a theoretical standpoint. However, its performance in the nonconvex context is less understood yet surprisingly impressive. This was particularly compelling to Jonathan M. Borwein who, intrigued by Elser, Rankenburg, and Thibault's success in applying the method for solving Sudoku Puzzles began an investigation of his own. We survey the current body of literature on the subject, and we summarize its history. We especially commemorate Professor Borwein's celebrated contributions to the area. arXiv:1809.07181v2 [math.OC]

show abstract

“…This observation allows us to apply the acceleration technique to affine settings beyond projectors including the Douglas-Rachford variants studied in [7,8,13,3]. The simplest realisation is the symmetrised Douglas-Rachford algorithm consider below.…”

Section: Extensions To Firmly Nonexpansive Operatorsmentioning

confidence: 99%

Gearhart-Koshy Acceleration for Affine Subspaces

Tam¹

2020

Preprint

View full text Add to dashboard Cite

The method of cyclic projection is an iterative scheme which can be used to find nearest points in the intersection of finitely many affine subspaces, having access only to the orthogonal projectors onto the individual subspaces. In an attempt to accelerate its convergence, Gearhart & Koshy (1989) proposed a modification of the scheme which, in each iteration, performs an exact line search based on minimising the distance to the desired nearest point. In the special case when the affine subspaces are linear subspaces, this line search can be made explicit by using the fact that the zero vector is always feasible. In this work, we derive an alternate (but nevertheless still explicit) implementation of this line search which does not require knowledge of a feasible vector, thus providing an efficient implementation of the scheme for affine subspaces.

show abstract

New Douglas--Rachford Algorithmic Structures and Their Convergence Analyses

Cited by 13 publications

References 26 publications

Deep Neural Network Structures Solving Variational Inequalities

Deep Neural Network Structures Solving Variational Inequalities

Survey: Sixty Years of Douglas–rachford

Gearhart-Koshy Acceleration for Affine Subspaces

Contact Info

Product

Resources

About