Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization

Nakayama, Shummin; Narushima, Y.; Yabe, Hiroshi

doi:10.3934/jimo.2018122

Cited by 10 publications

(9 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we only dealt with the memoryless BFGS formula for scaled proximal mappings. However, following previous research [21,20], we expect that the proposed algorithm based on the memoryless Broyden family with preconvex class ((25) with φ k < 0) may perform better than that with the memoryless BFGS formula. In deed, ( 25) can be rewritten as…”

Section: Discussionmentioning

confidence: 71%

Inexact proximal DC Newton-type method for nonconvex composite functions

Nakayama

Narushima

Yabe

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider a class of difference-of-convex (DC) optimization problems where the objective function is the sum of a smooth function and a possible nonsmooth DC function. The application of proximal DC algorithms to address this problem class is well-known. In this paper, we combine a proximal DC algorithm with an inexact proximal Newton-type method to propose an inexact proximal DC Newton-type method. We demonstrate global convergence properties of the proposed method. In addition, we give a memoryless quasi-Newton matrix for scaled proximal mappings and consider a two-dimensional system of semi-smooth equations that arise in calculating scaled proximal mappings. To efficiently obtain the scaled proximal mappings, we adopt a semi-smooth Newton method to inexactly solve the system. Finally, we present some numerical experiments to investigate the efficiency of the proposed method, showing that the proposed method outperforms existing methods.

show abstract

Section: Discussionmentioning

confidence: 71%

Inexact proximal DC Newton-type method for nonconvex composite functions

Nakayama

Narushima

Yabe

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Becker et al [5] gave a theorem similar to Theorem 2.2 that can be applied to the memoryless BFGS formula. However, L becomes more complicated than (23) (in this case, L is a two-dimensional function) and the exact solution of L(α) = 0 cannot be computed even if h(x) = λ∥x∥ 1 . In addition, in this case, L is not necessarily strongly monotone and its merit functions are possibly nondifferentiable.…”

Section: Algorithm 1 (Proximal Memoryless Sr1 Method)mentioning

confidence: 99%

“…Memoryless quasi-Newton methods are efficient especially for solving large-scale problems because the methods have lower memory requirements and use matrix-free computation. For that reason, there has been significant development of these methods (see, [19,20,22,23] and references therein). Since the SR1 method is an efficient method, some researchers have extended the method to memoryless SR1 methods [19,22].…”

mentioning

confidence: 99%

A proximal quasi-Newton method based on memoryless modified symmetric rank-one formula

Narushima

Nakayama

2023

JIMO

Self Cite

View full text Add to dashboard Cite

<p style='text-indent:20px;'>We consider proximal gradient methods for minimizing a composite function of a differentiable function and a convex function. To accelerate the general proximal gradient methods, we focus on proximal quasi-Newton type methods based on proximal mappings scaled by quasi-Newton matrices. Although it is usually difficult to compute the scaled proximal mappings, applying the memoryless symmetric rank-one (SR1) formula makes this easier. Since the scaled (quasi-Newton) matrices must be positive definite, we develop an algorithm using the memoryless SR1 formula based on a modified spectral scaling secant condition. We give the subsequential convergence property of the proposed method for general objective functions. In addition, we show the R-linear convergence property of the method under a strong convexity assumption. Finally, some numerical results are reported.</p>

show abstract

“…Shanno [27] proposed a memoryless quasi-Newton method as a way to deal with this problem. This method [9,[12][13][14][15] has proven effective at solving large-scale unconstrained optimization problems. The concept is simple: an approximate matrix is updated by using the identity matrix instead of the previous approximate matrix.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, he showed that the search direction is a sufficient descent direction and has the global convergence property. Nakayama, Narushima, and Yabe [15] proposed memoryless quasi-Newton methods based on the spectral-scaling Broyden family [3]. Their methods generate a sufficient descent direction and have the global convergence property.…”

Section: Introductionmentioning

confidence: 99%

Modified Memoryless Spectral-Scaling Broyden Family on Riemannian Manifolds

Sakai,

Iiduka

2024

J Optim Theory Appl

View full text Add to dashboard Cite

This paper presents modified memoryless quasi-Newton methods based on the spectral-scaling Broyden family on Riemannian manifolds. The method involves adding one parameter to the search direction of the memoryless self-scaling Broyden family on the manifold. Moreover, it uses a general map instead of vector transport. This idea has already been proposed within a general framework of Riemannian conjugate gradient methods where one can use vector transport, scaled vector transport, or an inverse retraction. We show that the search direction satisfies the sufficient descent condition under some assumptions on the parameters. In addition, we show global convergence of the proposed method under the Wolfe conditions. We numerically compare it with existing methods, including Riemannian conjugate gradient methods and the memoryless spectral-scaling Broyden family. The numerical results indicate that the proposed method with the BFGS formula is suitable for solving an off-diagonal cost function minimization problem on an oblique manifold.

show abstract

Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization

Cited by 10 publications

References 28 publications

Inexact proximal DC Newton-type method for nonconvex composite functions

Inexact proximal DC Newton-type method for nonconvex composite functions

A proximal quasi-Newton method based on memoryless modified symmetric rank-one formula

Modified Memoryless Spectral-Scaling Broyden Family on Riemannian Manifolds

Contact Info

Product

Resources

About