Inertial primal-dual algorithms for structured convex optimization

Chan, Raymond H.; Ma, Shiqian; Yang, Junfeng

doi:10.48550/arxiv.1409.2992

Cited by 2 publications

(1 citation statement)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the past decades, owing to the tremendous success of inertial acceleration [59,11], the inertial technique has been widely adapted to accelerate other first-order algorithms. For example inertial Douglas-Rachford and alternating direction method of multipliers (ADMM) [15,62,41,36], inertial Primal-Dual splitting [25,45]. In terms of Krasnosel'skiȋ-Mann fixed-point iteration, the inertial of it are also studied in the literature such as [53,30].…”

Section: Related Workmentioning

confidence: 99%

Geometry of First-Order Methods and Adaptive Acceleration

Poon¹,

Liang²

2020

Preprint

View full text Add to dashboard Cite

First-order operator splitting methods are ubiquitous among many fields through science and engineering, such as inverse problems, signal/image processing, statistics, data science and machine learning, to name a few. In this paper, we study a geometry property of first-order methods when applying to solve non-smooth optimization problems. With the tool "partial smoothness", we design a framework to analyze the trajectory of the fixed-point sequence generated by first-order methods and show that locally the fixed-point sequence settles onto a regular trajectory such as straight line or spiral. Based on this finding, we discuss the limitation of current widely used "inertial acceleration" technique, and propose a trajectory following adaptive acceleration algorithm. Global convergence is established for the proposed acceleration scheme based on the perturbation of fixed-point iteration. Locally, we first build connections between the acceleration scheme and the well studied "vector extrapolation technique" in the field of numerical analysis, and then discuss acceleration guarantees of the proposed acceleration scheme. Moreover, our result provides a geometric interpretation of vector extrapolation. Numeric experiments on various first-order methods are provided to demonstrate the advantage of the proposed adaptive acceleration scheme.

show abstract

Section: Related Workmentioning

confidence: 99%

Geometry of First-Order Methods and Adaptive Acceleration

Poon¹,

Liang²

2020

Preprint

View full text Add to dashboard Cite

show abstract

A note on the inertial proximal point method

Mu¹,

Yang²

2015

Stat., optim. inf. comput.

View full text Add to dashboard Cite

The proximal point method (PPM) for solving maximal monotone operator inclusion problem is a highly powerful tool for algorithm design, analysis and interpretation. To accelerate convergence of the PPM, inertial PPM (iPPM) was proposed in the literature. In this note, we show that some of the attractive properties of the PPM, e.g., the generated sequence is contractive with the set of solutions, do not hold in general for iPPM. To partially inherit the advantages of the PPM and meanwhile incorporate inertial extrapolation steps, we propose an iPPM with alternating inertial steps. Our analyses show that the even subsequence generated by the proposed iPPM is contractive with the set of solutions. Moreover, we establish global convergence result under much relaxed conditions on the inertial extrapolation stepsizes, e.g., monotonicity is no longer needed and the stepsizes are significantly enlarged compared to existing methods. Furthermore, we establish certain nonasymptotic O(1/k) and asymptotic o(1/k) convergence rate results, where k denotes the iteration counter. These features are new to inertial type PPMs.

show abstract

Inertial primal-dual algorithms for structured convex optimization

Cited by 2 publications

References 42 publications

Geometry of First-Order Methods and Adaptive Acceleration

Geometry of First-Order Methods and Adaptive Acceleration

A note on the inertial proximal point method

Contact Info

Product

Resources

About