Two new Hager–Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing

Abstract

Notwithstanding its efficiency and nice attributes, most research on the iterative scheme by Hager and Zhang [Pac. J. Optim. 2(1) (2006) 35-58] are focused on unconstrained minimization problems. Inspired by this and recent works by Waziri et al. [Appl. Math. Comput. 361(2019) 645-660], Sabi’u et al. [Appl. Numer. Math. 153(2020) 217-233], and Sabi’u et al. [Int. J. Comput. Meth, doi:10.1142/S0219876220500437], this paper extends the Hager-Zhang (HZ) approach to nonlinear monotone systems with convex constraint. Two new HZ-type iterative methods are developed by combining the prominent projection method by Solodov and Svaiter [Springer, pp 355-369, 1998] with HZ-type search directions, which are obtained by developing two new parameter choices for the Hager-Zhang scheme. The first choice, is obtained by minimizing the condition number of a modified HZ direction matrix, while the second choice is realized using singular value analysis and minimizing the spectral condition number of the nonsingular HZ search direction matrix. Interesting properties of the schemes include solving non-smooth functions and generating descent directions. Using standard assumptions, the methods’ global convergence are obtained and numerical experiments with recent methods in the literature, indicate that the methods proposed are promising. The schemes effectiveness are further demonstrated by their applications to sparse signal and image reconstruction problems, where they outperform some recent schemes in the literature.

A survey on the Dai–Liao family of nonlinear conjugate gradient methods

“…Nonlinear systems of equations often straightly emerge in engineering applications [58,103,114,116]. Especially, monotone cases of the model appear as the subproblems of the generalized proximal algorithms with Bregman distances [61].…”

“…As an initial study on the issue, Abubakar and Kumam [1] applied the DL method given in [20] to solve the problem. Then, based on eigenvalue analyses, Waziri et al [100,103] developed several DL methods with the parameter choices in the framework of HZ [55,57] for solving the nonlinear systems of monotone equations. Especially, in [103] they assessed efficiency of their algorithms in compressed sensing.…”

“…Then, based on eigenvalue analyses, Waziri et al [100,103] developed several DL methods with the parameter choices in the framework of HZ [55,57] for solving the nonlinear systems of monotone equations. Especially, in [103] they assessed efficiency of their algorithms in compressed sensing. Waziri et al [101,102] also used the DL algorithms given in [13,30,70,107] to address the model.…”

“…So, smooth relaxations of the image restoration model have been devised in the literature that can be addressed by large scale optimization algorithms [111]. As already mentioned, Babaie-Kafaki et al [6,31], Lu et al [79,80], Waziri et al [103] and Branislav et al [62] tackled the impulse noise removal of the images by the CG algorithms of the DL family.…”

On signal recovery and blurry image reconstruction via a double parameter Hager-Zhang type iterative scheme for monotone nonlinear equations

“…• Few number of modified HZ type schemes exists for problem (1.14) and the constrained variant. And the ones that exists are either modified versions of (1.9), as presented in [3,4] or modified adaptations of (1.10) as presented in [11,20,[24][25][26].…”