A nonmonotone trust region method based on simple conic models for unconstrained optimization

In this paper, we focus on the problem of blind source separation (BSS). To solve the problem efficiently, a new algorithm is proposed. First, a parallel variable dual-matrix model (PVDMM) that considers all the numerical relations between a mixing matrix and a separating matrix is proposed. Different constrained terms are used to construct the cost function for every sub-algorithm. These constrained terms reflect a numerical relation. Therefore, a number of undesired solutions are excluded, the search region is reduced, and the convergence efficiency of the algorithm is ultimately improved. Parallel sub-algorithms are proven to converge to a separable matrix only if the cost function approaches zero. Second, a descent algorithm (DA) as a PVDMM optimizing approach is proposed in this paper as well. Apparently, the efficiency of the algorithm depends completely on the DA. Unlike the traditional descent method, the DA defines step length by solving inequality instead of merely utilizing the Wolfeor Armijo-type search rule. Stimulation results indicate that the DA can improve computational efficiency. Under mild conditions, the DA has been proven to have strong convergence properties. Numerical results also show that the method is very efficient and robust. Finally, we applied the combinative algorithm to the BSS problem. Computer simulations illustrate its good performance.

show abstract

“…The derivation of cost functions gradient and Hessian matrix is based on the following properties [21]: …”

Section: Appendix: the Derivation Of Gradient And Hessian Matrix Of Cmentioning

confidence: 99%

“…Without a doubt, the iterative optimization algorithm plays an important role in whole process of solving the BSS problem. Below is the expression of the unconstrained optimization problem [12,19,21].…”

Section: Introductionmentioning

confidence: 99%

A Descent Algorithm-Based Parallel Variable Dual Matrix Model and Application to Blind Source Separation

Zeng

Feng

2014

Circuits Syst Signal Process

View full text Add to dashboard Cite

show abstract

“…In the next experiments, we compare Algorithm NCGL with three-term CG method (MDL) [18], inexact Newton method [6] and Trust region method (NSCTR) [19], respectively. The numerical results are listed in Tables 2, 3 and 4.…”

Section: Numerical Experimentsmentioning

confidence: 99%

A Nonmonotone Hybrid Method of Conjugate Gradient and Lanczos-type for Solving Nonlinear Systems

Chen

Wang

Zhu

2014

J. Oper. Res. Soc. China

View full text Add to dashboard Cite

In this paper, we construct a new algorithm which combines the conjugate gradient and Lanczos methods for solving nonlinear systems. The iterative direction can be obtained by solving a quadratic model via conjugate gradient and Lanczos methods. Using the backtracking line search, we will find an acceptable trial step size along this direction which makes the objective function nonmonotonically decreasing and makes the norm of the step size monotonically increasing. Global convergence and local superlinear convergence rate of the proposed algorithm are established under some reasonable conditions. Finally, we present some numerical results to illustrate the effectiveness of the proposed algorithm. Keywords Nonmonotonic technique Á Nonlinear systems Á Lanczos method Á Conjugate gradient The authors gratefully acknowledge the partial supports of the National Natural Science Foundation of China (No. 11371253).

show abstract

“…for which f : R n −→ R is a continuously differentiable function and bounded from below. There are many kinds of iterative methods to solve unconstrained optimization problems including the Newton method [28], the steepest descent method [29], the conjugate gradient methods [11,13,16,23], the quasi-Newton methods [9,[18][19][20], line search methods [9,22] and trust-region methods [26,34,37,40]. Conjugate gradient methods [13,16,22,24,30] are the powerful classes of iterative methods to solve unconstrained optimization problems, which are suitable especially for the large-scale problems due to the simplicity of their iterates and low memory requirements.…”

Section: Introductionmentioning

confidence: 99%

Extended Dai-Yuan conjugate gradient strategy for large-scale unconstrained optimization with applications to compressive sensing

Esmaeili

Rostami

Kimiaei

2018

Filomat

View full text Add to dashboard Cite

We present a new spectral conjugate gradient method based on the Dai-Yuan strategy to solve large-scale unconstrained optimization problems with applications to compressive sensing. In our method, the numerator of conjugate gradient parameter is a convex combination from the maximum gradient norm value in some preceding iterates and the current gradient norm value. This combination will try to produce the larger step-size far away from the optimizer and the smaller step-size close to it. In addition, the spectral parameter guarantees the descent property of the new generated direction in each iterate. The global convergence results are established under some standard assumptions. Numerical results are reported which indicate the promising behavior of the new procedure to solve large-scale unconstrained optimization and compressive sensing problems.

show abstract

A nonmonotone trust region method based on simple conic models for unconstrained optimization

Cited by 5 publications

References 15 publications

A Descent Algorithm-Based Parallel Variable Dual Matrix Model and Application to Blind Source Separation

A Descent Algorithm-Based Parallel Variable Dual Matrix Model and Application to Blind Source Separation

A Nonmonotone Hybrid Method of Conjugate Gradient and Lanczos-type for Solving Nonlinear Systems

Extended Dai-Yuan conjugate gradient strategy for large-scale unconstrained optimization with applications to compressive sensing

Contact Info

Product

Resources

About