Data assimilation approach to analysing systems of ordinary differential equations

Osojnik, Ana; Cartis, Coralia; Madho, Godwin; Jones, C. A.; Tobias, Steven M.

doi:10.1109/iscas.2018.8351751

Cited by 4 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Implementation A Python implementation of DFBGN is available on Github. 3 Notation We use • to refer to the Euclidean norm of vectors and the operator 2-norm of matrices, and B(x, ∆) for x ∈ R n and ∆ > 0 to be the closed ball {y ∈ R n : y − x ≤ ∆}.…”

Section: Structure Of Papermentioning

confidence: 99%

“…Existing model-based DFO techniques are primarily designed for small-to medium-scale problems, as the linear algebra cost of each iteration-largely due to the cost of constructing interpolation models-means that their runtime increases rapidly for large problems. There are several settings where scalable DFO algorithms may be useful, such as data assimilation [9,3], machine learning [67,35], generating adversarial examples for deep neural networks [2,68], image analysis [30], and as a possible proxy for global optimization methods [19].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Cartis¹,

Roberts²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce a general framework for large-scale model-based derivative-free optimization based on iterative minimization within random subspaces. We present a probabilistic worst-case complexity analysis for our method, where in particular we prove high-probability bounds on the number of iterations before a given optimality is achieved. This framework is specialized to nonlinear least-squares problems, with a model-based framework based on the Gauss-Newton method. This method achieves scalability by constructing local linear interpolation models to approximate the Jacobian, and computes new steps at each iteration in a subspace with user-determined dimension. We then describe a practical implementation of this framework, which we call DFBGN. We outline efficient techniques for selecting the interpolation points and search subspace, yielding an implementation that has a low per-iteration linear algebra cost (linear in the problem dimension) while also achieving fast objective decrease as measured by evaluations. Extensive numerical results demonstrate that DFBGN has improved scalability, yielding strong performance on large-scale nonlinear least-squares problems.

show abstract

Section: Structure Of Papermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Cartis¹,

Roberts²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We then describe a practical implementation of RSDFO-GN, which we call DFBGN (Derivative-Free Block Gauss-Newton). 3 Compared to existing methods, DFBGN reduces the linear algebra cost of model construction and the initial objective evaluation cost by allowing fewer interpolation points at every iteration. In order for DFBGN to have both scalability and a similar evaluation efficiency to existing methods (i.e.…”

Section: Contributionsmentioning

confidence: 99%

“…Existing model-based DFO techniques are primarily designed for small-to medium-scale problems, as the linear algebra cost of each iteration-largely due to the cost of constructing interpolation models-means that their runtime increases rapidly for large problems. There are several settings where scalable DFO algorithms may be useful, such as data assimilation [3,10], machine learning [39,71], generating adversarial examples for deep neural networks [2,75], image analysis [34], and as a possible proxy for global optimization methods [21].…”

Section: Introductionmentioning

confidence: 99%

Scalable subspace methods for derivative-free nonlinear least-squares optimization

Cartis

Roberts

2022

Math. Program.

Self Cite

View full text Add to dashboard Cite

We introduce a general framework for large-scale model-based derivative-free optimization based on iterative minimization within random subspaces. We present a probabilistic worst-case complexity analysis for our method, where in particular we prove high-probability bounds on the number of iterations before a given optimality is achieved. This framework is specialized to nonlinear least-squares problems, with a model-based framework based on the Gauss–Newton method. This method achieves scalability by constructing local linear interpolation models to approximate the Jacobian, and computes new steps at each iteration in a subspace with user-determined dimension. We then describe a practical implementation of this framework, which we call DFBGN. We outline efficient techniques for selecting the interpolation points and search subspace, yielding an implementation that has a low per-iteration linear algebra cost (linear in the problem dimension) while also achieving fast objective decrease as measured by evaluations. Extensive numerical results demonstrate that DFBGN has improved scalability, yielding strong performance on large-scale nonlinear least-squares problems.

show abstract

A derivative-free Gauss–Newton method

Cartis

Roberts

2019

Math. Prog. Comp.

Self Cite

View full text Add to dashboard Cite

We present DFO-GN, a derivative-free version of the Gauss-Newton method for solving nonlinear least-squares problems. As is common in derivative-free optimization, DFO-GN uses interpolation of function values to build a model of the objective, which is then used within a trust-region framework to give a globally-convergent algorithm requiring O( −2 ) iterations to reach approximate first-order criticality within tolerance . This algorithm is a simplification of the method from [H. Zhang, A. R. Conn, and K. Scheinberg, A Derivative-Free Algorithm for Least-Squares Minimization, SIAM J. Optim., 20 (2010), pp. 3555-3576], where we replace quadratic models for each residual with linear models. We demonstrate that DFO-GN performs comparably to the method of Zhang et al. in terms of objective evaluations, as well as having a substantially faster runtime and improved scalability.

show abstract

Data assimilation approach to analysing systems of ordinary differential equations

Cited by 4 publications

References 8 publications

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Scalable subspace methods for derivative-free nonlinear least-squares optimization

A derivative-free Gauss–Newton method

Contact Info

Product

Resources

About