High-order differentiable autoencoder for nonlinear model reduction

Abstract

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton's method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., 10+ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable weight-varying Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.

Numerical Coarsening with Neural Shape Functions

“…[FMD*19] use an autoencoder to construct nonlinear subspaces automatically. Some algorithms have been used to separate the subspace apart and learn only the nonlinear corrections, such as those applied in [RCPO21] and [SYS*21]. Beyond model reduction, NNWarp in [LSW*18] makes use of simple networks to correct linear nodal deformation to nonlinear ones.…”

Subspace Mixed Finite Elements for Real-Time Heterogeneous Elastodynamics

“…These methods unfortunately scale in complexity with the number of rotations to be tracked, of which there may be many in a largescale heterogeneous material. While non-linear subspaces via Deep Neural Networks have also been proposed [Shen et al 2021], the resulting complexity of the subspace requires many optimization steps in order to reach a solution [Sharp et al 2023].…”

Voronoi Filters for Simulation Enrichment

“…Other works have also used ideas similar to kinematic filters to transfer data between representations, e.g, between grids and particles [JSS*15], or between grids and rigid modes [FLLP13]. Some recent works constrain motion to the null‐space of predefined reduced spaces; some use kinematic filters to implement the constraint efficiently [SYS*21, RCPO22], while others could replace Lagrange multiplier formulations with kinematic filters for higher performance [ZBLJ20].…”