Reduced Order Modeling using Shallow ReLU Networks with Grassmann Layers

Abstract

This paper presents a nonlinear model reduction method for systems of equations using a structured neural network. The neural network takes the form of a "three-layer" network with the first layer constrained to lie on the Grassmann manifold and the first activation function set to identity, while the remaining network is a standard two-layer ReLU neural network. The Grassmann layer determines the reduced basis for the input space, while the remaining layers approximate the nonlinear input-output system. The training alternates between learning the reduced basis and the nonlinear approximation, and is shown to be more effective than fixing the reduced basis and training the network only. An additional benefit of this approach is, for data that lie on low-dimensional subspaces, that the number of parameters in the network does not need to be large. We show that our method can be applied to scientific problems in the data-scarce regime, which is typically not well-suited for neural network approximations. Examples include reduced order modeling for nonlinear dynamical systems and several aerospace engineering problems.

Adaptive Projected Residual Networks for Learning Parametric Maps from Sparse Data

“…A recent topic of interest in scientific machine learning is the deployment of neural networks as parametric surrogates [4,6,13,20,22,24,31,32,38]. Of particular relevance to this work are projection based parametric neural surrogates which seek to parametrize high dimensional maps by use of linear and nonlinear dimension reduction strategies [4,6,13,31,38].…”