Stability of discrete-time feed-forward neural networks in NARX configuration

Bonassi, Fabio; Farina, Marcello; Scattolini, Riccardo

doi:10.1016/j.ifacol.2021.08.417

Cited by 30 publications

(40 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 1, which correspond to the states of the dynamical system representing the plant under study. A first simple RNN architecture is that of the so-called Neural Nonlinear ARX (NNARX) [49,50], widely employed mainly thanks to their simple structure and training. This architecture makes use of FFNN as regression functions embedded in a NARX setting, and it is trained minimizing the simulation error, thus obtaining a model able to mimic the system's free-run simulation.…”

Section: Nn For Reinforcement Learningmentioning

confidence: 99%

“…Model (4) can easily be recast in the general form (3). Specifically, (4) corresponds to a discrete-time normal canonical form [49]. Indeed, letting i P t1, ..., N u…”

Section: Nnarx Networkmentioning

confidence: 99%

“…Proposition 1. Under suitable conditions on their weights Φ, NNARXs [49], ESNs [54], LSTMs [18], and GRUs [60] are guaranteed to be ISS and δISS. These conditions, synthetically reported in the Appendix, can be generally regarded as nonlinear inequalities on the weights of the network, denoted by…”

Section: Network Training and Stabilitymentioning

confidence: 99%

“…Denoting by δu k the input perturbation, from (2) it follows that the resulting deviation of the state trajectory is bounded as }δx k } 2 ď γp}δu k } 2,8 q. It is worth noticing that function γ can be conservatively computed from the weights of the network, see [18,49,54,60]. However, if the RNN model is δISS, less conservative bounds on γ can be computed numerically on a sufficiently large set of model trajectories.…”

Section: Robustnessmentioning

confidence: 99%

See 3 more Smart Citations

On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments

Bonassi,

Farina,

Xie

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper aims to discuss and analyze the potentialities of Recurrent Neural Networks (RNN) in control design applications. The main families of RNN are considered, namely Neural Nonlinear AutoRegressive eXogenous, (NNARX), Echo State Networks (ESN), Long Short Term Memory (LSTM), and Gated Recurrent Units (GRU). The goal is twofold. Firstly, to survey recent results concerning the training of RNN that enjoy Input-to-State Stability (ISS) and Incremental Input-to-State Stability (δISS) guarantees. Secondly, to discuss the issues that still hinder the widespread use of RNN for control, namely their robustness, verifiability, and interpretability. The former properties are related to the so-called generalization capabilities of the networks, i.e. their consistency with the underlying real plants, even in presence of unseen or perturbed input trajectories. The latter is instead related to the possibility of providing a clear formal connection between the RNN model and the plant. In this context, we illustrate how ISS and δISS represent a significant step towards the robustness and verifiability of the RNN models, while the requirement of interpretability paves the way to the use of physics-based networks. The design of model predictive controllers with RNN as plant's model is also briefly discussed. Lastly, some of the main topics of the paper are illustrated on a simulated chemical system.

show abstract

Section: Nn For Reinforcement Learningmentioning

confidence: 99%

“…Model (4) can easily be recast in the general form (3). Specifically, (4) corresponds to a discrete-time normal canonical form [49]. Indeed, letting i P t1, ..., N u…”

Section: Nnarx Networkmentioning

confidence: 99%

Section: Network Training and Stabilitymentioning

confidence: 99%

Section: Robustnessmentioning

confidence: 99%

See 2 more Smart Citations

On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments

Bonassi,

Farina,

Xie

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…First, gated recurrent networks naturally retain long-term memory of the past trajectories. In contrast, for feed-forward auto-regressive architectures, this memory must be enforced by supplying the past input-output data-points as inputs of the network [24], typically resulting in less accurate long-term learning compared to RNNs [25]. Secondly, since the controller is implemented by a gated recurrent network, which is inherently strictly proper, the issue of the controller's improperness discussed in [24] does not arise.…”

Section: Introductionmentioning

confidence: 99%

Recurrent Neural Network-based Internal Model Control design for stable nonlinear systems

Bonassi,

Scattolini

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Owing to their superior modeling capabilities, gated Recurrent Neural Networks (RNNs), such as Gated Recurrent Units (GRUs) and Long Short-Term Memory networks (LSTMs), have become popular tools for learning dynamical systems. This paper aims to discuss how these networks can be adopted for the synthesis of Internal Model Control (IMC) architectures. To this end, a first gated RNN is used to learn a model of the unknown input-output stable plant. Then, another gated RNN approximating the model inverse is trained. The proposed scheme is able to cope with the saturation of the control variables, and it can be deployed on low-power embedded controllers since it does not require any online computation. The approach is then tested on the Quadruple Tank benchmark system, resulting in satisfactory closed-loop performances.

show abstract

Data Driven prediction of forced nonlinear vibrations using stabilised Autoregressive Neural Networks

Westmeier

Kreuter

Bäuerle

et al. 2023

Proc Appl Math and Mech

View full text Add to dashboard Cite

In this work, we propose a novel approach to the data‐driven prediction of vibration responses of nonlinear systems. The main idea is based on Autoregressive Neural Networks (ARNN) to model the nonlinear transfer behaviour between an external excitation and the system response. We propose an autoregressive network architecture with embedded symmetry using bias‐free tanh activation and guarantee Input‐to‐State‐Stability (ISS) by enforcing a special penalty term to the weights. The resulting training procedure is analysed for the example of a DUFFING oscillator with white noise excitation. In a BAYESian optimisation, it is found that beyond enforcing input‐to‐state‐stability, the stabilising penalty term also decreases sensitivity with respect to other training parameters compared to other classical techniques. Furthermore, we show that the stabilised ARNN is able to give excellent approximations of the nonlinear response of the DUFFING oscillator for a wide range of excitation intensities. In contrast, linear models, such as autoregressive models with exogenous input (ARX) in time domain or linear transfer functions in frequency domain, will only find some linear approximation. In particular, by construction, they will not be able to capture nonlinear effects for arbitrary amplitudes and excitation levels.

show abstract

Stability of discrete-time feed-forward neural networks in NARX configuration

Cited by 30 publications

References 33 publications

On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments

On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments

Recurrent Neural Network-based Internal Model Control design for stable nonlinear systems

Data Driven prediction of forced nonlinear vibrations using stabilised Autoregressive Neural Networks

Contact Info

Product

Resources

About