Performance enhancement of Gauss-Newton trust-region solver for distributed Gauss-Newton optimization method

Gao, Guohua; Jiang, Hao; Vink, Jeroen C.; Hagen, Paul P. H. van; Wells, Terence

doi:10.1007/s10596-019-09830-x

Cited by 5 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead of applying the Newton-Raphson method which requires evaluating ϕ ′ (λ), Gao et al [32] proposed a method to directly solve the TRS using inverse quadratic model interpolation (called the DIQ method), i.e., approximating π(λ) [s(λ)] T s(λ) by the following inverse quadratic function,…”

Section: The Inverse Quadratic Model Interpolation Methods To Directly Solve the L-bfgs Trsmentioning

confidence: 99%

“…To save both memory usage and computational cost, Gao, et al [32,36,37] proposed an efficient algorithm to solve the Gauss-Newton TRS (GNTRS) for large-scale history matching problems using the matrix inversion lemma (or the Woodbury matrix identity). With appropriate normalization of both parameters and residuals, the Hessian of the objective function for a history matching problem can be approximated by the well-known Gauss-Newton equation as follows,…”

Section: Using Matrix Inversion Lemma (Mil) To Solve the L-bfgs Trsmentioning

confidence: 99%

“…The GNTRS solver proposed by Gao, et al [32,36,37] using the matrix inversion lemma (MIL) has been implemented and integrated with the distributed Gauss-Newton (DGN) optimizer. Because the compact representation of the L-BFGS Hessian updating formula Eq.…”

Section: Using Matrix Inversion Lemma (Mil) To Solve the L-bfgs Trsmentioning

confidence: 99%

“…4 is similar to Eq. 15, we follow the similar idea as proposed by Gao, et al [32,36,37] to compute [H (k+1) + λI n ] − 1 by applying the matrix inversion lemma and then solve the L-BFGS trust region search step s(λ).…”

Section: Using Matrix Inversion Lemma (Mil) To Solve the L-bfgs Trsmentioning

confidence: 99%

“…For completeness, we discuss the compact representation of the L-BFGS Hessian updating formulation [31] and the algorithm to directly update the Hessian in the next section directly. In the third section, we present three different methods to solve the L-BFGS TRS: the DNR method, the direct method using inverse quadratic (DIQ) interpolation approach proposed by Gao et al [32], and the technique using matrix inversion lemma (MIL) together with an efficient matrix updating algorithm. Some numerical tests and performance comparisons are discussed in the fourth section.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Performance Analysis of Trust Region Subproblem Solvers for Limited-Memory Distributed BFGS Optimization Method

Gao

Florez

Vink

et al. 2021

Front. Appl. Math. Stat.

Self Cite

View full text Add to dashboard Cite

The limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) optimization method performs very efficiently for large-scale problems. A trust region search method generally performs more efficiently and robustly than a line search method, especially when the gradient of the objective function cannot be accurately evaluated. The computational cost of an L-BFGS trust region subproblem (TRS) solver depend mainly on the number of unknown variables (n) and the number of variable shift vectors and gradient change vectors (m) used for Hessian updating, with m << n for large-scale problems. In this paper, we analyze the performances of different methods to solve the L-BFGS TRS. The first method is the direct method using the Newton-Raphson (DNR) method and Cholesky factorization of a dense n × n matrix, the second one is the direct method based on an inverse quadratic (DIQ) interpolation, and the third one is a new method that combines the matrix inversion lemma (MIL) with an approach to update associated matrices and vectors. The MIL approach is applied to reduce the dimension of the original problem with n variables to a new problem with m variables. Instead of directly using expensive matrix-matrix and matrix-vector multiplications to solve the L-BFGS TRS, a more efficient approach is employed to update matrices and vectors iteratively. The L-BFGS TRS solver using the MIL method performs more efficiently than using the DNR method or DIQ method. Testing on a representative suite of problems indicates that the new method can converge to optimal solutions comparable to those obtained using the DNR or DIQ method. Its computational cost represents only a modest overhead over the well-known L-BFGS line-search method but delivers improved stability in the presence of inaccurate gradients. When compared to the solver using the DNR or DIQ method, the new TRS solver can reduce computational cost by a factor proportional to n2/m for large-scale problems.

show abstract

Section: The Inverse Quadratic Model Interpolation Methods To Directly Solve the L-bfgs Trsmentioning

confidence: 99%

Section: Using Matrix Inversion Lemma (Mil) To Solve the L-bfgs Trsmentioning

confidence: 99%

Section: Using Matrix Inversion Lemma (Mil) To Solve the L-bfgs Trsmentioning

confidence: 99%

Section: Using Matrix Inversion Lemma (Mil) To Solve the L-bfgs Trsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Performance Analysis of Trust Region Subproblem Solvers for Limited-Memory Distributed BFGS Optimization Method

Gao

Florez

Vink

et al. 2021

Front. Appl. Math. Stat.

Self Cite

View full text Add to dashboard Cite

show abstract

Distributed quasi-Newton derivative-free optimization method for optimization problems with multiple local optima

et al. 2021

View full text Add to dashboard Cite

Strategies to Enhance the Performance of Gaussian Mixture Model Fitting for Uncertainty Quantification by Conditioning to Production Data

Gao

Vink

Saaf

et al. 2021

Day 1 Tue, October 26, 2021

View full text Add to dashboard Cite

When formulating history matching within the Bayesian framework, we may quantify the uncertainty of model parameters and production forecasts using conditional realizations sampled from the posterior probability density function (PDF). It is quite challenging to sample such a posterior PDF. Some methods e.g., Markov chain Monte Carlo (MCMC), are very expensive (e.g., MCMC) while others are cheaper but may generate biased samples. In this paper, we propose an unconstrained Gaussian Mixture Model (GMM) fitting method to approximate the posterior PDF and investigate new strategies to further enhance its performance. To reduce the CPU time of handling bound constraints, we reformulate the GMM fitting formulation such that an unconstrained optimization algorithm can be applied to find the optimal solution of unknown GMM parameters. To obtain a sufficiently accurate GMM approximation with the lowest number of Gaussian components, we generate random initial guesses, remove components with very small or very large mixture weights after each GMM fitting iteration and prevent their reappearance using a dedicated filter. To prevent overfitting, we only add a new Gaussian component if the quality of the GMM approximation on a (large) set of blind-test data sufficiently improves. The unconstrained GMM fitting method with the new strategies proposed in this paper is validated using nonlinear toy problems and then applied to a synthetic history matching example. It can construct a GMM approximation of the posterior PDF that is comparable to the MCMC method, and it is significantly more efficient than the constrained GMM fitting formulation, e.g., reducing the CPU time by a factor of 800 to 7300 for problems we tested, which makes it quite attractive for large scale history matching problems.

show abstract

Performance enhancement of Gauss-Newton trust-region solver for distributed Gauss-Newton optimization method

Cited by 5 publications

References 33 publications

Performance Analysis of Trust Region Subproblem Solvers for Limited-Memory Distributed BFGS Optimization Method

Performance Analysis of Trust Region Subproblem Solvers for Limited-Memory Distributed BFGS Optimization Method

Distributed quasi-Newton derivative-free optimization method for optimization problems with multiple local optima

Strategies to Enhance the Performance of Gaussian Mixture Model Fitting for Uncertainty Quantification by Conditioning to Production Data

Contact Info

Product

Resources

About