Forward sensitivity approach for estimating eddy viscosity closures in nonlinear model reduction

Ahmed, Shady E.; Bhar, Kinjal; San, Omer; Rasheed, Adil

doi:10.1103/physreve.102.043302

Cited by 6 publications

(3 citation statements)

References 120 publications

(126 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our current work, we formulate an RL framework to learn , and call this approach as multi-modal RL (MMRL). Although the proposed closure problem can be formulated using more traditional adjoint based 37 or sensitivity based approaches 42 , our chief motivation in this study is to explore the feasiblity of RL workflows for the ROM closure problems. More specifically, in this paper we aim to introduce a variational multiscale RL (VMRL) approach by formulating a new procedure to forge a reward function that does not require access to the training data.…”

Section: Methodsmentioning

confidence: 99%

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

San

Pawar

Rasheed

2022

Sci Rep

View full text Add to dashboard Cite

A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement learning (RL) is a powerful yet relatively uncharted method in spatiotemporally extended systems. In this study, we put forth a modular dynamic closure modeling and discovery framework to stabilize the Galerkin projection based reduced order models that may arise in many nonlinear spatiotemporal dynamical systems with quadratic nonlinearity. However, a key element in creating a robust RL agent is to introduce a feasible reward function, which can be constituted of any difference metrics between the RL model and high fidelity simulation data. First, we introduce a multi-modal RL to discover mode-dependant closure policies that utilize the high fidelity data in rewarding our RL agent. We then formulate a variational multiscale RL (VMRL) approach to discover closure models without requiring access to the high fidelity data in designing the reward function. Specifically, our chief innovation is to leverage variational multiscale formalism to quantify the difference between modal interactions in Galerkin systems. Our results in simulating the viscous Burgers equation indicate that the proposed VMRL method leads to robust and accurate closure parameterizations, and it may potentially be used to discover scale-aware closure models for complex dynamical systems.

show abstract

Section: Methodsmentioning

confidence: 99%

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

San

Pawar

Rasheed

2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…In our current work, we formulate an RL framework to learn η k (t), and call this approach as multi-modal RL (MMRL). Although the proposed closure problem can be formulated using more traditional adjoint based 37 or sensitivity based approaches 42 , our chief motivation in this study is to explore the feasiblity of RL workflows for the ROM closure problems. More specifically, in this paper we aim to introduce a variational multiscale RL (VMRL) approach by formulating a new procedure to forge a reward function that does not require access to the training data.…”

Section: Closure Modelingmentioning

confidence: 99%

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

San¹,

Pawar²,

Rasheed³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement learning (RL) is a powerful yet relatively uncharted method in spatiotemporally extended systems. In this study, we put forth a modular dynamic closure modeling and discovery framework to stabilize the Galerkin projection based reduced order models that may arise in many nonlinear spatiotemporal dynamical systems with quadratic nonlinearity. However, a key element in creating a robust RL agent is to introduce a feasible reward function, which can be constituted of any difference metrics between the RL model and high fidelity simulation data. First, we introduce a multi-modal RL (MMRL) to discover mode-dependant closure policies that utilize the high fidelity data in rewarding our RL agent. We then formulate a variational multiscale RL (VMRL) approach to discover closure models without requiring access to the high fidelity data in designing the reward function. Specifically, our chief innovation is to leverage variational multiscale formalism to quantify the difference between modal interactions in Galerkin systems. Our results in simulating the viscous Burgers equation indicate that the proposed VMRL method leads to robust and accurate closure parameterizations, and it may potentially be used to discover scale-aware closure models for complex dynamical systems.

show abstract

“…This is the first of the two advantages referred to in the title of this paper. The papers by Lewis et al [26,27] and Ahmed et al [28,29] contain several applications of this strategy.…”

Section: Introductionmentioning

confidence: 99%

On the dual advantage of placing observations through forward sensitivity analysis

et al. 2023

Self Cite

View full text Add to dashboard Cite

The four-dimensional variational data assimilation methodology for assimilating noisy observations into a deterministic model has been the workhorse in forecasting centres for over three decades. While this method provides a computationally efficient framework for dynamic data assimilation, it is largely silent on the important question concerning the minimum number and placement of observations. To answer this question, we demonstrate the dual advantage of placing the observations where the square of the sensitivity of the model solution with respect to the unknown control variables, called forward sensitivities, attains its maximum. By following this approach, we can force the observability Gramian to be of full rank, which in turn guarantees efficient recovery of the optimal values of the control variables, which is the first of the two advantages of this strategy. We further show that the proposed strategy of placing observations has another inherent optimality: the square of the sensitivity of the optimal estimates of the control with respect to the observations (used to obtain these estimates) attains its minimum value, a second advantage that is a direct consequence of the above strategy for placing observations. Our analytical framework and numerical experiments on linear and nonlinear systems confirm the effectiveness of our proposed strategy.

show abstract

Forward sensitivity approach for estimating eddy viscosity closures in nonlinear model reduction

Cited by 6 publications

References 120 publications

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

On the dual advantage of placing observations through forward sensitivity analysis

Contact Info

Product

Resources

About