Surrogate Modeling of Nonlinear Dynamic Systems: A Comparative Study

Abstract

Surrogate models play a vital role in overcoming the computational challenge in designing and analyzing nonlinear dynamic systems, especially in the presence of uncertainty. This paper presents a comparative study of different surrogate modeling techniques for nonlinear dynamic systems. Four surrogate modeling methods, namely Gaussian process (GP) regression, a long short-term memory (LSTM) network, a convolutional neural network (CNN) with LSTM (CNN-LSTM), and a CNN with bidirectional LSTM (CNN-BLSTM), are studied and compared. All these model types can predict future behavior of dynamic systems over long periods based on training data from relatively short periods. The multi-dimensional inputs of surrogate models are organized in a nonlinear autoregressive exogenous model (NARX) scheme to enable recursive prediction over long periods, where current predictions replace inputs from the previous time window. Three numerical examples, including one mathematical example and two nonlinear engineering analysis models, are used to compare the performance of the four surrogate modeling techniques. The results show that the GP-NARX surrogate model tends to have more stable performance than the other three deep learning-based methods for the three particular examples studied. The tuning effort of GP-NARX is also much lower than its deep learning-based counterparts.

Data Driven Level Set Fuzzy and Neural Modeling

“…In particular, a scenario comprising 195 training instances subject to normally distributed noise N (0, 0.1) was considered to train long short-term memories (LSTM), convolutional neural networks with long short-term memory (CNN-LSTM), convolutional neural networks with bidirectional long short-term memory (CNN-BLSTM), and Gaussian process nonlinear autoregressive exogenous models (GP-NARX). The results of [13] show that, the average of 104 testing instances, that LSTM, CNN-LSTM, and CNN-BLSTM achieve essentially similar mean square errors. GP-NARX achieved the worst performance in average.…”

“…The experimental setup to evaluate the LSTM, CNN, DLSM, and the RLSM is as follows. In the vein of the data generation process and experiments done in the literature [13], a dataset with 302 instances is produced using (12), from which 200 are noisy instances with N (0, 0.1) used for training, and the remaining 102 noisefree instances are used for testing. The level set fuzzy models have six rules with Gaussian membership functions.…”

“…More recent work addresses deep models for system identification [12]. Additionally, nonlinear autoregressive Gaussian processes with exogenous inputs, long short-term memory, and convolutional neural network to model nonlinear dynamic systems have been explored [13].…”

A data‐driven Bayesian approach for optimal dynamic product transitions

“…The surrogate model is then optimized, and its predictions are used to guide the search for the optimal solution. 57,58 • Reinforcement learning (RL): RL techniques, such as Q-learning or deep reinforcement learning, optimize decision-making processes through trial and error learning. RL can be employed for optimizing complex systems where actions influence future states and outcomes.…”

“… Surrogate‐based optimization: This approach involves constructing a surrogate model (e.g., GPs, neural networks) based on the available data to approximate the objective function. The surrogate model is then optimized, and its predictions are used to guide the search for the optimal solution 57,58 Reinforcement learning (RL): RL techniques, such as Q‐learning or deep reinforcement learning, optimize decision‐making processes through trial and error learning.…”