On the adaptation of recurrent neural networks for system identification

Abstract

This paper presents a transfer learning approach which enables fast and efficient adaptation of Recurrent Neural Network (RNN) models of dynamical systems. A nominal RNN model is first identified using available measurements. The system dynamics are then assumed to change, leading to an unacceptable degradation of the nominal model performance on the perturbed system. To cope with the mismatch, the model is augmented with an additive correction term trained on fresh data from the new dynamic regime. The correction term is learned through a Jacobian Feature Regression (JFR) method defined in terms of the features spanned by the model's Jacobian with respect to its nominal parameters. A non-parametric view of the approach is also proposed, which extends recent work on Gaussian Process (GP) with Neural Tangent Kernel (NTK-GP) to the RNN case (RNTK-GP). This can be more efficient for very large networks or when only few data points are available. Implementation aspects for fast and efficient computation of the correction term, as well as the initial state estimation for the RNN model are described. Numerical examples show the effectiveness of the proposed methodology in presence of significant system variations.

Adaptation of Dynamic Data‐Driven Models for Real‐Time Applications: From Simulated to Real Batch Distillation Trajectories by Transfer Learning

“…The use of TL from simulated to real plant data (sim2real) was explored by [18][19][20], but has not been applied to chemical processes. Little has been reported on applications of TL for complex dynamic systems in the chemical industry.…”

“…3) or even adding new layers on top of the RNN (d in Fig. 3) [19] are virtually unlimited and are not in the scope of this contribution.…”