Data-Driven Polynomial Ridge Approximation Using Variable Projection

Abstract

Inexpensive surrogates are useful for reducing the cost of science and engineering studies involving large-scale, complex computational models with many input parameters. A ridge approximation is one class of surrogate that models a quantity of interest as a nonlinear function of a few linear combinations of the input parameters. When used in parameter studies (e.g., optimization or uncertainty quantification), ridge approximations allow the low-dimensional structure to be exploited, reducing the effective dimension. We introduce a new, fast algorithm for constructing a ridge approximation where the nonlinear function is a polynomial. This polynomial ridge approximation is chosen to minimize least squared mismatch between the surrogate and the quantity of interest on a given set of inputs. Naively, this would require optimizing both the polynomial coefficients and the linear combination of weights; the latter of which define a low-dimensional subspace of the input space. However, given a fixed subspace the optimal polynomial can be found by solving a linear least-squares problem. Hence using variable projection the polynomial can be implicitly defined, leaving an optimization problem over the subspace alone. Here we develop an algorithm that finds this polynomial ridge approximation by minimizing over the Grassmann manifold of low-dimensional subspaces using a Gauss-Newton method. Our Gauss-Newton method has superior theoretical guarantees and faster convergence on our numerical examples than the alternating approach for polynomial ridge approximation earlier proposed by Constantine, Eftekhari, Hokanson, and Ward [https://doi.org/10.1016/j.cma.2017.07.038] that alternates between (i) optimizing the polynomial coefficients given the subspace and (ii) optimizing the subspace given the coefficients.

Dimension reduction in heterogeneous parametric spaces with application to naval engineering shape design problems

“…Active subspaces can be seen in the more general context of ridge approximation (see [35,26]). In particular it can be proved that, under certain conditions, the active subspace is nearly stationary and it is a good starting point in optimal ridge approximation as shown in [12,23].…”

Supporting multi-point fan design with dimension reduction

“…Note that (14) corresponds to a total order index set, where the sum of the composite univariate polynomial orders in the multivariate expansion is less than or equal to k. Such a basis was used in Ref. (25), resulting in a total of 351 coefficients (setting d = 25 and k = 2 in (14)) that needed to be estimated. Once obtained, computing gradients of this global quadratic is trivial, resulting in the following estimate of the covariance matrix…”

“…Motivated by the variable projection aided dimension reduction approach in Ref. (14), we devote Section 4 to its study, offering a few minor algorithmic variations. Then, we test this on one of our blades, being parsimonious in the number of samples (and hence CFD evaluations) used.…”

Inverse regression for ridge recovery: a data-driven approach for parameter reduction in computer experiments

“…Note that the ridge recovery problem is distinct from ridge approximation (Constantine et al, 2017b;Hokanson and Constantine, 2017), where the goal is to find A and construct g that minimize the approximation error for a given f . 1 Inverse regression may be useful for ridge approximation or identifying near-ridge structure in a given function, but pursuing these ideas is beyond the scope of this manuscript.…”