No abstract

In this paper, we present a method for identifying infeasible, unbounded, and pathological conic programs based on Douglas-Rachford splitting, or equivalently ADMM. When an optimization program is infeasible, unbounded, or pathological, the iterates of Douglas-Rachford splitting diverge. Somewhat surprisingly, such divergent iterates still provide useful information, which our method uses for identification. In addition, for strongly infeasible problems the method produces a separating hyperplane and informs the user on how to minimally modify the given problem to achieve strong feasibility. As a first-order method, the proposed algorithm relies on simple subroutines, and therefore is simple to implement and has low per-iteration cost.Keywords Douglas-Rachford Splitting · infeasible, unbounded, pathological, conic programs 1 IntroductionMany convex optimization algorithms have strong theoretical guarantees and empirical performance, but they are often limited to non-pathological, feasible problems; under pathologies often the theory breaks down and the empirical performance degrades significantly. In fact, the behavior of convex optimization algorithms under pathologies has been studied much less, and many existing solvers often simply report "failure" without informing the users of what went wrong upon encountering infeasibility, unboundedness, or pathology. Pathological problem are numerically challenging, but they are not impossible to deal with. As infeasibility, unboundedness, and pathology do arise in practice (see, for example, [17,16]), designing a robust algorithm that behaves well in all cases is important to the completion of a robust solver.In this paper, we propose a method based on Douglas-Rachford splitting (DRS), or equivalently ADMM, that identifies infeasible, unbounded, and pathological conic programs. First-order methods such as DRS/ADMM are simple and can quickly provide a solution with moderate accuracy. It is well known, for example, by combining Theorem 1 of [29] and Proposition 4.4 of [12], that the iterates of DRS/ADMM converge to a fixed point if there is one (a fixed point z * of an operator T satisfies z * = T z * ), and when there is no fixed point, the iterates diverge unboundedly. However, the precise manner in which they diverge has been studied much less. Somewhat surprisingly, when iterates of DRS/ADMM diverge, the behavior of the iterates still provides useful information, which we use to classify the conic program. For example, a separating hyperplane can be found when the conic program is strongly infeasible, and an improving direction can be obtained when there is one. When the problem is infeasible or weakly feasible, it is useful to know how to minimally modify the problem data to achieve strong feasibility. We also get this information via the divergent iterates.

Despite the vast literature on DRS and ADMM, there has been very little work analyzing their behavior under pathologies. Most analyses assume a primal solution exists, a dual solution exists, and strong duality holds. When these assumptions are not met, i.e., under pathologies, the theory often breaks down and the empirical performance may degrade significantly. In this paper, we establish that DRS only requires strong duality to work, in the sense that asymptotically iterates are approximately feasible and approximately optimal.Keywords Douglas-Rachford splitting · Strong Duality · Pathological convex programs Mathematics Subject Classification (2010) 90C46 · 49N15 · 90C25 1 Introduction Douglas-Rachford splitting (DRS) and alternating directions method of multipliers (ADMM) are classical methods originally presented in [61,35,49,47] and [44,46], respectively. DRS and ADMM are closely related. Over the last decade, these methods have enjoyed a resurgence of popularity, as the demand to solve ever larger problems grew.DRS and ADMM have strong theoretical guarantees and empirical performance, but such results are often limited to non-pathological problems; in Ernest K. Ryu UCLA

No abstract

It is known that operator splitting methods based on Forward Backward Splitting (FBS), Douglas-Rachford Splitting (DRS), and Davis-Yin Splitting (DYS) decompose a difficult optimization problems into simpler subproblem under proper convexity and smoothness assumptions. In this paper, we identify an envelope (an objective function) whose gradient descent iteration under a variable metric coincides with DYS iteration. This result generalizes the Moreau envelope for proximal-point iteration and the envelopes for FBS and DRS iterations identified by Patrinos, Stella, and Themelis.Based on the new envelope and the Stable-Center Manifold Theorem, we further show that, when FBS or DRS iterations start from random points, they avoid all strict saddle points with probability one. This result extends the similar results by Lee et al. from gradient descent to splitting methods. 4When h = 0, (5) simplifies to Douglas-Rachford Splitting iteration,When g = 0, P γg reduces to Id and thus (5) simplifies towhich is Forward-Backward Splitting iteration slightly generalized by including the linear operator L.When f = 0, P γf reduces to Id and (5) simplifies to Backward-Forward Splitting,When f = g = 0, (5) simplifies to gradient descent iteration Derivation of envelopeNow we show that, (5) can be written as gradient descent iteration of an envelope function under the following assumption.Assumption 1.