Levenberg--Marquardt Methods Based on Probabilistic Gradient Models and Inexact Subproblem Solution, with Application to Data Assimilation

Bergou, El Houcine; Gratton, Serge; Vicente, Luís Nunes

doi:10.1137/140974687

Cited by 32 publications

(34 citation statements)

References 19 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…for some θ 2 ∈ 0, 1 2 , achieves the Cauchy decrease (2.2), with θ = 2(1−θ 2 ) ∈ [1,2). Proof For the proof see [5], Lemma 4.1.…”

Section: )mentioning

confidence: 93%

“…The result of this minimization procedure is the initial state of a dynamical system, which is then integrated forward in time to produce a weather forecast. This topic has been studied for example in [5,15,16]. In [15] conjugate-gradients methods for the solution of nonlinear least-squares problems regularized by a quadratic penalty term are investigated.…”

Section: Introductionmentioning

confidence: 99%

“…In [16] an observation-thinning method for the efficient numerical solution of large-scale incremental four dimensional (4D-Var) data assimilation problems is proposed, built exploiting an adaptive hierarchy of the observations which are successively added based on a posteriori error estimate. In [5] a Levenberg-Marquardt approach is proposed to deal with random gradient and Jacobian models. It is assumed that an approximation to the gradient is provided but only accurate with a certain probability and the knowledge of the probability of the error between the exact and the approximated gradient is used in the update of the regularization parameter.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper we present a modification of the approach proposed in [5], to obtain a method able to take into account inaccuracy also in the objective function, while in [5] it is assumed to have at disposal the exact function values. In our procedure, following [11] and deviating from [5], we replace the request made in [5] on first-order accuracy of the gradient up to a certain probability with a control on the accuracy level of the function values. Then, we propose a Levenberg-Marquardt approach that aims at finding a solution of problem (1.1) considering a sequence of approximations f δ k of known and increasing accuracy.…”

Section: Introductionmentioning

confidence: 99%

“…We are not aware of Levenberg-Marquardt methods for both zero and non-zero residual nonlinear least-squares problems with approximated function and gradient, for which both local and global convergence is proved. Contributions on this topic are given by [5] where the inexactness is present only in the gradient and in the Jacobian and local convergence is not proved and by [3,4] where the Jacobian is exact and only local convergence is considered.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients

2018

View full text Add to dashboard Cite

This is an author's version published in: http://oatao.univ-toulouse.fr/24842To cite this version: Bellavia, Stefania and Gratton, Serge and Riccietti, Elisa A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients. (2018) Numerische Mathematik, 140 (3). 791-825. ISSN 0029-599X Official URL: https://doi. AbstractIn this paper we consider large scale nonlinear least-squares problems for which function and gradient are evaluated with dynamic accuracy and propose a Levenberg-Marquardt method for solving such problems. More precisely, we consider the case in which the exact function to optimize is not available or its evaluation is computationally demanding, but approximations of it are available at any prescribed accuracy level. The proposed method relies on a control of the accuracy level, and imposes an improvement of function approximations when the accuracy is detected to be too low to proceed with the optimization process. We prove global and local convergence and complexity of our procedure and show encouraging numerical results on test problems arising in data assimilation and machine learning. Mathematics Subject ClassificationAchieving the Cauchy decrease is a sufficient condition to get global convergence of the method, so one can rely on approximated solutions of problem (2.1), see [11]. A solution of (2.1) can alternatively be found solving the optimality conditions(2.3)Then, we approximately solve (2.3), i.e. we compute a step p such that J δ k (x k ) T J δ k (x k ) + λ k I p = −g δ k (x k ) + r k ,

show abstract

“…for some θ 2 ∈ 0, 1 2 , achieves the Cauchy decrease (2.2), with θ = 2(1−θ 2 ) ∈ [1,2). Proof For the proof see [5], Lemma 4.1.…”

Section: )mentioning

confidence: 93%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients

2018

View full text Add to dashboard Cite

show abstract

Numerical linear algebra in data assimilation

Freitag

2020

GAMM-Mitteilungen

View full text Add to dashboard Cite

Data assimilation is a method that combines observations (ie, real world data) of a state of a system with model output for that system in order to improve the estimate of the state of the system and thereby the model output. The model is usually represented by a discretized partial differential equation. The data assimilation problem can be formulated as a large scale Bayesian inverse problem. Based on this interpretation we will derive the most important variational and sequential data assimilation approaches, in particular three‐dimensional and four‐dimensional variational data assimilation (3D‐Var and 4D‐Var) and the Kalman filter. We will then consider more advanced methods which are extensions of the Kalman filter and variational data assimilation and pay particular attention to their advantages and disadvantages. The data assimilation problem usually results in a very large optimization problem and/or a very large linear system to solve (due to inclusion of time and space dimensions). Therefore, the second part of this article aims to review advances and challenges, in particular from the numerical linear algebra perspective, within the various data assimilation approaches.

show abstract

Outsmarting the Atmospheric Turbulence for Ground-Based Telescopes Using the Stochastic Levenberg-Marquardt Method

Hong

Bergou

Doucet

et al. 2021

Euro-Par 2021: Parallel Processing

View full text Add to dashboard Cite

One of the main challenges for ground-based optical astronomy is to compensate for atmospheric turbulence in near real-time. The goal is to obtain images as close as possible to the diffraction limit of the telescope. This challenge is addressed on the latest generation of giant optical telescopes by deploying multi-conjugate adaptive optics (MCAO) systems performing predictive tomography of the turbulence and multilayer compensation. Such complex systems require a high fidelity estimate of the turbulence profile above the telescope, to be updated regularly during operations as turbulence conditions evolve. In this paper, we modify the traditional Levenberg-Marquardt (LM) algorithm by considering stochastically chosen subsystems of the full problem to identify the required parameters efficiently, while coping with the real-time challenge. While LM operates on the full set data samples, the resulting Stochastic LM (SLM) method randomly selects subsamples to compute corresponding approximate gradients and Hessians. Hence, SLM reduces the algorithmic complexity per iteration and shortens the overall time to solution, while maintaining LM's numerical robustness. We present a new convergence analysis for SLM, implement the algorithm with optimized GPU kernels, and deploy it on shared-memory systems with multiple GPU accelerators. We assess SLM in the adaptive optics system configurations in the context of the MCAO-Assisted Visible Imager & Spectrograph (MAVIS) instrument for the Very Large Telescope (VLT). We demonstrate performance superiority of SLM over the traditional LM algorithm and the classical stochastic first-order methods. At the scale of VLT AO, SLM finishes the optimization process and accurately retrieves the parameters (e.g., turbulence strength and wind speed profiles) in less than a second using up to eight NVIDIA A100 GPUs, which permits high acuity real-time throughput over a night of observations.

show abstract

Levenberg--Marquardt Methods Based on Probabilistic Gradient Models and Inexact Subproblem Solution, with Application to Data Assimilation

Cited by 32 publications

References 19 publications

A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients

A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients

Numerical linear algebra in data assimilation

Outsmarting the Atmospheric Turbulence for Ground-Based Telescopes Using the Stochastic Levenberg-Marquardt Method

Contact Info

Product

Resources

About