New quasi-Newton methods via higher order tensor models

Biglari, Fahimeh; Hassan, Musa Abu; Leong, Wah June

doi:10.1016/j.cam.2010.10.041

Cited by 33 publications

(22 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a proof of these statements, see Theorem 1 in [5] and Theorem 2.1 in [29]. Now, using (7)-(9), one can easily realize that, for the case s k > 1, the standard QN equation and Wei's modified QN equation have the same order of accuracy in presence of the Tensor and both perform better than the Biglari's modified QN equation.…”

Section: Some Bfgs-type Methodsmentioning

confidence: 88%

“…Now, using (7)-(9), one can easily realize that, for the case s k > 1, the standard QN equation and Wei's modified QN equation have the same order of accuracy in presence of the Tensor and both perform better than the Biglari's modified QN equation. Moreover, using (5) and (6), the standard QN equation is more accurate than the Zhang's modified QN equation whenever s k > 1.…”

Section: Some Bfgs-type Methodsmentioning

confidence: 96%

“…Wei et al [29] proposed another modified QN equation that can be obtained by replacing θ k with 1 3 θ k in (4). Recently, Biglari et al [5] derived a new modified QN equation based on the fourth order Taylor formula for the objective function by replacing θ k with 2 3 θ k in (4). They showed that the BFGS formula obtained from the new proposed equation is superior to the standard BFGS and that of obtained form the QN equation proposed by Wei et al [29].…”

Section: Some Bfgs-type Methodsmentioning

confidence: 99%

“…[12,25]. Indeed, it requires some modifications on its structure to be used for general functions [5,20,29,32,33]. On the other hand, some regularization techniques have been introduced in the literature to construct global convergence property of quasi-Newton methods for the general functions.…”

Section: Introductionmentioning

confidence: 98%

See 3 more Smart Citations

A new regularized limited memory BFGS-type method based on modified secant conditions for unconstrained optimization problems

Tarzanagh

Peyghami

2015

J Glob Optim

View full text Add to dashboard Cite

In the framework of large-scale optimization problems, the standard BFGS method is not affordable due to memory constraints. The so-called limited-memory BFGS (L-BFGS) method is an adaption of the BFGS method for large-scale settings. However, the standard BFGS method and therefore the standard L-BFGS method only use the gradient information of the objective function and neglect function values. In this paper, we propose a new regularized L-BFGS method for solving large scale unconstrained optimization problems in which more available information from the function and gradient values are employed to approximate the curvature of the objective function. The proposed method utilizes a class of modified quasi-Newton equations in order to achieve higher order accuracy in approximating the second order curvature of the objective function. Under some standard assumptions, we provide the global convergence property of the new method. In order to provide an efficient method for finding global minima of a continuously differentiable function, a hybrid algorithm that combines a genetic algorithm (GA) with the new proposed regularized L-BFGS method has been proposed. This combination leads the iterates to a stationary point of the objective function with higher chance of being global minima. Numerical results show the efficiency and robustness of the new proposed regularized L-BFGS and its hybridized version with GA in practice.

show abstract

Section: Some Bfgs-type Methodsmentioning

confidence: 88%

Section: Some Bfgs-type Methodsmentioning

confidence: 96%

Section: Some Bfgs-type Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

See 2 more Smart Citations

A new regularized limited memory BFGS-type method based on modified secant conditions for unconstrained optimization problems

Tarzanagh

Peyghami

2015

J Glob Optim

View full text Add to dashboard Cite

show abstract

“…Numerical experiments show that their method seems to be better. Based on the work in [3], Biglari and Solimanpur [4] recently proposed four choices of β k by considering a fourth order conic model applied to the objective function:…”

Section: Smooth Bound Constrained Optimizationmentioning

confidence: 99%

Learning Unknown Structure in CRFs via Adaptive Gradient Projection Method

Xue

Zhang

2016

Stat., optim. inf. comput.

View full text Add to dashboard Cite

This paper focuses on learning unknown structure in conditional random fields (CRFs), especially learning both the structure and parameters of a CRF model simultaneously. By adding the l 2 -regularization to node parameters and the group l 1 -regularization to edge parameters, this structure learning problem can be cast as a convex minimization problem. Then an adaptive gradient method is proposed to solve the minimization problem. Extensive simulation experiments are presented to show the performance of the proposed approach for learning unknown structure.

show abstract