Error bounds and convergence analysis of feasible descent methods: a general approach

Luo, Ze; Tseng, Paul

doi:10.1007/bf02096261

Cited by 340 publications

(298 citation statements)

References 51 publications

Supporting

Mentioning

296

Contrasting

Order By: Relevance

“…By Lemmas 2.3 and 2.2, the triangle inequality, and (8), it follows that For i>i 1 Letting i-> oo, we get that {f(x i )} -»-oo, which contradicts the fact that f( •) is bounded from below on X. Hence, the assumption is invalid, and lt i->oo {x i } n J5f (<p, 0)^0.…”

Section: ( •) On R( •) and E( •) We Establish A Certain Relationshipmentioning

confidence: 93%

“…Combining the latter inequality with (11), we further obtain where the second relation follows from the Cauchy-Schwartz inequality, the third inequality follows from the definition of e(. ), and the last inequality follows from (10) for i sufficiently large, say i>i 1 .…”

Section: ( •) On R( •) and E( •) We Establish A Certain Relationshipmentioning

confidence: 97%

“…Following Ref. 1. we consider a broad class of feasible descent methods that can be represented by the formula where n is a positive scalar and the mapping e: R n+1 ->R n is the defining feature of each particular algorithm; see Section 3.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Convergence Analysis of Perturbed Feasible Descent Methods

Solodov

1997

Journal of Optimization Theory and Applications

View full text Add to dashboard Cite

Abstract. We develop a general approach to convergence analysis of feasible descent methods in the presence of perturbations. The important novel feature of our analysis is that perturbations need not tend to zero in the limit. In that case, standard convergence analysis techniques are not applicable. Therefore, a new approach is needed. We show that, in the presence of perturbations, a certain e-approximate solution can be obtained, where e depends linearly on the level of perturbations. Applications to the gradient projection, proximal minimization, extragradient and incremental gradient algorithms are described.

show abstract

Section: ( •) On R( •) and E( •) We Establish A Certain Relationshipmentioning

confidence: 93%

Section: ( •) On R( •) and E( •) We Establish A Certain Relationshipmentioning

confidence: 97%

See 1 more Smart Citation

Convergence Analysis of Perturbed Feasible Descent Methods

Solodov

1997

Journal of Optimization Theory and Applications

View full text Add to dashboard Cite

show abstract

“…Assumption 2(a) is a local Lipschitzian error bound assumption, saying that the distance from x to X * is locally in the order of the norm of the residual at x; see [24] and references therein. Assumption 2(b) says that the isocost surfaces of F ρ restricted to the solution set X * are "properly separated."…”

Section: Lemma 32 For Any H 0 N and Nonemptymentioning

confidence: 99%

“…Assumption 2(b) says that the isocost surfaces of F ρ restricted to the solution set X * are "properly separated." Assumption 2(b) holds automatically if f is convex or f is quadratic; see [24,39] for further discussions. Upon applying [39,Theorem 4] to the problem (1), we obtain the following sufficient conditions for Assumption 2(a) to hold.…”

Section: Lemma 32 For Any H 0 N and Nonemptymentioning

confidence: 99%

A coordinate gradient descent method for ℓ 1-regularized convex minimization

Yun

Toh

2009

Comput Optim Appl

View full text Add to dashboard Cite

In applications such as signal processing and statistics, many problems involve finding sparse solutions to under-determined linear systems of equations. These problems can be formulated as a structured nonsmooth optimization problems, i.e., the problem of minimizing 1 -regularized linear least squares problems. In this paper, we propose a block coordinate gradient descent method (abbreviated as CGD) to solve the more general 1 -regularized convex minimization problems, i.e., the problem of minimizing an 1 -regularized convex smooth function. We establish a Q-linear convergence rate for our method when the coordinate block is chosen by a Gauss-Southwell-type rule to ensure sufficient descent. We propose efficient implementations of the CGD method and report numerical results for solving large-scale 1 -regularized linear least squares problems arising in compressed sensing and image deconvolution as well as large-scale 1 -regularized logistic regression problems for feature selection in data classification. Comparison with several state-of-the-art algorithms specifically designed for solving large-scale 1 -regularized linear least squares or logistic regression problems suggests that an efficiently implemented CGD method may outperform these algorithms despite the fact that the CGD method is not specifically designed just to solve these special classes of problems.

show abstract

Capacity expansion planning through augmented Lagrangian optimization and scenario decomposition

Ierapetritou

2011

AIChE Journal

View full text Add to dashboard Cite

Stochastic programming is a typical method for addressing the uncertainties in capacity expansion planning problem. However, the corresponding deterministic equivalent model is often intractable with considerable number of uncertainty scenarios especially for stochastic integer programming (SIP) based formulations. In this article, a hybrid solution framework consisting of augmented Lagrangian optimization and scenario decomposition algorithm is proposed to solve the SIP problem. The method divides the solution procedure into two phases, where traditional linearization based decomposition strategy and global optimization technique are applied to solve the relaxation problem successively. Using the proposed solution framework, a feasible solution of the original problem can be obtained after the first solution phase whereas the optimal solution is obtained after the second solution phase. The effectiveness of the proposed strategy is verified through a numerical example of two stage stochastic integer program and the capacity expansion planning examples.

show abstract

Error bounds and convergence analysis of feasible descent methods: a general approach

Cited by 340 publications

References 51 publications

Convergence Analysis of Perturbed Feasible Descent Methods

Convergence Analysis of Perturbed Feasible Descent Methods

A coordinate gradient descent method for ℓ 1-regularized convex minimization

Capacity expansion planning through augmented Lagrangian optimization and scenario decomposition

Contact Info

Product

Resources

About