The quasigradient method for the solving of the nonlinear programming problems

Nurminskii, E. A.

doi:10.1007/bf01068677

Cited by 39 publications

(36 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The class of weakly convex functions, first introduced in English in [50], is broad. It includes all convex functions and smooth functions with Lipschitz continuous gradient.…”

mentioning

confidence: 99%

Stochastic Model-Based Minimization of Weakly Convex Functions

Davis¹,

2019

View full text Add to dashboard Cite

We consider a family of algorithms that successively sample and minimize simple stochastic models of the objective function. We show that under reasonable conditions on approximation quality and regularity of the models, any such algorithm drives a natural stationarity measure to zero at the rate O(k −1/4 ). As a consequence, we obtain the first complexity guarantees for the stochastic proximal point, proximal subgradient, and regularized Gauss-Newton methods for minimizing compositions of convex functions with smooth maps. The guiding principle, underlying the complexity guarantees, is that all algorithms under consideration can be interpreted as approximate descent methods on an implicit smoothing of the problem, given by the Moreau envelope. Specializing to classical circumstances, we obtain the long-sought convergence rate of the stochastic projected gradient method, without batching, for minimizing a smooth function on a closed convex set.

show abstract

“…The class of weakly convex functions, first introduced in English in [50], is broad. It includes all convex functions and smooth functions with Lipschitz continuous gradient.…”

mentioning

confidence: 99%

Stochastic Model-Based Minimization of Weakly Convex Functions

Davis¹,

2019

View full text Add to dashboard Cite

show abstract

“…In [12], Nurminski considered the case of nonccnvex nondifferentiable functions Fb(x) which satisfy the inequality The conditions of this theorem are similar to necessary and sufficient convergence conditions proposed by Zangwill [51]. However, Zangwill's conditions are very difficult to verify for a nonmonotonic procedure.…”

Section: Sandg Methods For Nonconvex Functionsmentioning

confidence: 87%

“…We could also use ys instead of y(x" ws) (see (16)). It is easy to see that (17) and (18) we can also use approximations (1 1) or (12) to compute the gradient gx (for a differentiable function g(., y, w")).…”

Section: ~' ( X )mentioning

confidence: 99%

stochastic quasigradient methods and their application to system optimization^†

1983

View full text Add to dashboard Cite

This paper systematically surveys the development of stochastic quasigradient (SQG) methods. These methods make it possible to solve optimization problems without calculating the precise values of objectives and constraints (let alone of their derivatives). For deterministic nonlinear optimization problems, these methods can be regarded as methods of random search. For stochastic programming problems, SQG methods generalize the wellknown stochastic approximation method for unconstrained optimization of the expectation of a random function to problems involving general constraints.

show abstract

“…Next we introduce the definition of a µ-relative weakly convex function, given in [32]. This definition extends the definition of weakly convexity [33], which was employed in the analysis of nonconvex optimization methods.…”

Section: Smooth Adaptable Function and Relative Weakly Convexitymentioning

confidence: 99%

Bregman Proximal Gradient Algorithm With Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

et al. 2019

View full text Add to dashboard Cite

In this paper, we consider an accelerated method for solving nonconvex and nonsmooth minimization problems. We propose a Bregman Proximal Gradient algorithm with extrapolation(BPGe). This algorithm extends and accelerates the Bregman Proximal Gradient algorithm (BPG), which circumvents the restrictive global Lipschitz gradient continuity assumption needed in Proximal Gradient algorithms (PG). The BPGe algorithm has higher generality than the recently introduced Proximal Gradient algorithm with extrapolation(PGe), and besides, due to the extrapolation step, BPGe converges faster than BPG algorithm. Analyzing the convergence, we prove that any limit point of the sequence generated by BPGe is a stationary point of the problem by choosing parameters properly. Besides, assuming Kurdyka-Lojasiewicz property, we prove the whole sequences generated by BPGe converges to a stationary point. Finally, to illustrate the potential of the new method BPGe, we apply it to two important practical problems that arise in many fundamental applications (and that not satisfy global Lipschitz gradient continuity assumption): Poisson linear inverse problems and quadratic inverse problems. In the tests the accelerated BPGe algorithm shows faster convergence results, giving an interesting new algorithm.

show abstract

The quasigradient method for the solving of the nonlinear programming problems

Cited by 39 publications

References 0 publications

Stochastic Model-Based Minimization of Weakly Convex Functions

Stochastic Model-Based Minimization of Weakly Convex Functions

stochastic quasigradient methods and their application to system optimization^†

Bregman Proximal Gradient Algorithm With Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

Contact Info

Product

Resources

About

The quasigradient method for the solving of the nonlinear programming problems

Cited by 39 publications

References 0 publications

Stochastic Model-Based Minimization of Weakly Convex Functions

Stochastic Model-Based Minimization of Weakly Convex Functions

stochastic quasigradient methods and their application to system optimization†

Bregman Proximal Gradient Algorithm With Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

Contact Info

Product

Resources

About

stochastic quasigradient methods and their application to system optimization^†