Iteratively regularized Gauss–Newton type methods for approximating quasi–solutions of irregular nonlinear operator equations in Hilbert space with an application to COVID–19 epidemic dynamics

Kokurin, M. Yu.; Kokurin, M. Yu.; Семенова, А. В.

doi:10.1016/j.amc.2022.127312

Cited by 4 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the learning relations in (8), we obtain machine learning algorithms for estimating the rows of the corresponding matrices in the form of the process in (13). Consequently, the question of analyzing the quality of algorithms for updating matrices in QNMs will consist of analyzing learning relations like (8) and the degree of orthogonality of the vectors involved in training.…”

Section: Matrix Learning Algorithms In Quasi-newton Methodsmentioning

confidence: 99%

“…Let the current approximation H of the matrix H* = A −1 be known. It is required to construct a new approximation using the learning relations in (7) for the rows of the matrix in (8):…”

Section: Gradient Learning Algorithms For Deriving and Analyzing Matr...mentioning

confidence: 99%

“…Quasi-Newton minimization methods are effective tools of solving smooth minimization problems when the function level curves have a high degree of elongation [4][5][6][7]. QNMs are commonly applied in a wide range of areas, such as biology [8], image processing [9], technics [10][11][12][13][14][15], and deep learning [16][17][18].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning in Quasi-Newton Methods

Krutikov,

Tovbis,

Stanimirović

et al. 2024

Axioms

View full text Add to dashboard Cite

In this article, we consider the correction of metric matrices in quasi-Newton methods (QNM) from the perspective of machine learning theory. Based on training information for estimating the matrix of the second derivatives of a function, we formulate a quality functional and minimize it by using gradient machine learning algorithms. We demonstrate that this approach leads us to the well-known ways of updating metric matrices used in QNM. The learning algorithm for finding metric matrices performs minimization along a system of directions, the orthogonality of which determines the convergence rate of the learning process. The degree of learning vectors’ orthogonality can be increased both by choosing a QNM and by using additional orthogonalization methods. It has been shown theoretically that the orthogonality degree of learning vectors in the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is higher than in the Davidon–Fletcher–Powell (DFP) method, which determines the advantage of the BFGS method. In our paper, we discuss some orthogonalization techniques. One of them is to include iterations with orthogonalization or an exact one-dimensional descent. As a result, it is theoretically possible to detect the cumulative effect of reducing the optimization space on quadratic functions. Another way to increase the orthogonality degree of learning vectors at the initial stages of the QNM is a special choice of initial metric matrices. Our computational experiments on problems with a high degree of conditionality have confirmed the stated theoretical assumptions.

show abstract

Section: Matrix Learning Algorithms In Quasi-newton Methodsmentioning

confidence: 99%

“…Let the current approximation H of the matrix H* = A −1 be known. It is required to construct a new approximation using the learning relations in (7) for the rows of the matrix in (8):…”

Section: Gradient Learning Algorithms For Deriving and Analyzing Matr...mentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning in Quasi-Newton Methods

Krutikov,

Tovbis,

Stanimirović

et al. 2024

Axioms

View full text Add to dashboard Cite

show abstract

“…In microscopy, QN methods help to achieve high resolution imaging [4]. In modeling the spread of infections, QN is useful for the identification of the unknown model coefficients [5]. QN methods are also useful for the modeling of complex crack propagation [6], fluid-structure interaction [7][8][9], melting and solidification of alloys [10], heat transfer systems [11], etc.…”

Section: Introductionmentioning

confidence: 99%

On the Convergence Rate of Quasi-Newton Methods on Strongly Convex Functions with Lipschitz Gradient

Krutikov,

Tovbis,

Stanimirović

et al. 2023

Mathematics

View full text Add to dashboard Cite

The main results of the study of the convergence rate of quasi-Newton minimization methods were obtained under the assumption that the method operates in the region of the extremum of the function, where there is a stable quadratic representation of the function. Methods based on the quadratic model of the function in the extremum area show significant advantages over classical gradient methods. When solving a specific problem using the quasi-Newton method, a huge number of iterations occur outside the extremum area, unless there is a stable quadratic approximation of the function. In this paper, we study the convergence rate of quasi-Newton-type methods on strongly convex functions with a Lipschitz gradient, without using local quadratic approximations of a function based on the properties of its Hessian. We proved that quasi-Newton methods converge on strongly convex functions with a Lipschitz gradient with the rate of a geometric progression, while the estimate of the convergence rate improves with the increasing number of iterations, which reflects the fact that the learning (adaptation) effect accumulates as the method operates. Another important fact discovered during the theoretical study is the ability of quasi-Newton methods to eliminate the background that slows down the convergence rate. This elimination is achieved through a certain linear transformation that normalizes the elongation of function level surfaces in different directions. All studies were carried out without any assumptions regarding the matrix of second derivatives of the function being minimized.

show abstract

“…Traditional models for the spread of infectious diseases are the so-called compartment models, including the SIS (susceptible-infected-susceptible) and the SIR (susceptible-infected-recovered) models [5] , [6] , [7] , [8] , or the more sophisticated SIRS (susceptible-infected-recovered-susceptible) and SEIR (susceptible-exposed-infected-recovered) models [9] , [10] , [11] , [12] , among many others. In these models, individuals in the population are divided into different compartments that describe their status (susceptible or infected, for example).…”

Section: Introductionmentioning

confidence: 99%