General parameterized proximal point algorithm with applications in statistical learning

Abstract

In the literature, there are a few researches to design some parameters in the Proximal Point Algorithm (PPA), especially for the multi-objective convex optimizations. Introducing some parameters to PPA can make it more flexible and attractive. Mainly motivated by our recent work (Bai et al., A parameterized proximal point algorithm for separable convex optimization, Optim. ), in this paper we develop a general parameterized PPA with a relaxation step for solving the multi-block separable structured convex programming. By making use of the variational inequality and some mathematical identities, the global convergence and the worst-case O(1/t) convergence rate of the proposed algorithm are established. Preliminary numerical experiments on solving a sparse matrix minimization problem from statistical learning validate that our algorithm is more efficient than several state-of-the-art algorithms.

Generalized symmetric ADMM for separable convex optimization

“…We have investigated the effects of the stepsizes (τ, s) and the penal parameter β on the performance of GS-ADMM. And our numerical experiments demonstrate that by properly choosing the parameters, GS-ADMM could perform significantly better than other recently quite popular methods developed in [1,14,17,23].…”

“…functions f i and g j in (1). As a result, in many recent real applications involving big data, solving the subproblems of ALM becomes very expensive.…”

“…where f i (x i ) : R mi → R, g j (y j ) : R dj → R are closed and proper convex functions (possibly nonsmooth); A i ∈ R n×mi , B j ∈ R n×dj and c ∈ R n are given matrices and vectors, respectively; X i ⊂ R mi and Y j ⊂ R dj are closed convex sets; p ≥ 1 and q ≥ 1 are two integers. Throughout this paper, we assume that the solution set of the problem (1) is nonempty and all the matrices A i , i = 1, • • • , p, and B j , j = 1, • • • , q, have full column rank. And in the following, we denote…”

A Proximal Point Approach for Distributed System State Estimation

“…The major contribution of this work consists in the design of a distributed iterative procedure for system state estimation. Its novelty relies on the use of Proximal Point (PP) method, nowadays widely employed in different research areas (such as statistical learning (Bai et al, 2019)) and recently exploited in the development of algorithms for the solution of estimation tasks resting upon the LS approach (Aster et al, 2018). A deep insight is provided on the role of topological links that define the communication constraints in the system w.r.t.…”

Convergence Analysis on an Accelerated Proximal Point Algorithm for Linearly Constrained Optimization Problems

“…In some practical problems with suitable conditions, the proximal point subproblems may be expressed as convex optimization problems with smaller scales or better properties which make them even have closed form solutions. In recent years, the proximal point method, together with its relative models, and several types of PPAs have been used in machine learning, image recognition, signal processing, and so on [1][2][3].…”