Deep Convolutional Networks in System Identification

Andersson, Carl; Ribeiro, Antônio H.; Tiels, Koen; Wahlström, Niklas; Schön, Thomas B.

doi:10.1109/cdc40024.2019.9030219

Cited by 52 publications

(36 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…From a practical perspective, the effectiveness of deep CNN architectures for different system identification and time series modeling tasks has been demonstrated in several contributions. [3][4][5] Even though mathematically it is not clear whether increasing the number of hidden layers extends the class of dynamics that can be represented by CNNs, experimentally it has been observed that deeper networks are able to learn more complex dependencies than shallower ones, for a given number of training parameters and for a given computational effort.…”

Section: Representational Power Of Dynonet Architecturesmentioning

confidence: 99%

“…Among the layers routinely applied in DL, 1D convolution 3 is the closest match. In particular, the 1D causal convolution layer 4,5 corresponds to the filtering of an input sequence through a causal finite impulse response (FIR) dynamical system. The dynoNet architecture may be seen as a generalization of the causal 1D convolutional neural network (CNN) enabling IIR filtering, owing to the description of the dynamical layers as rational transfer functions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

dynoNet: A neural network architecture for learning dynamical systems

Forgione

Piga

2021

Adaptive Control & Signal

View full text Add to dashboard Cite

This article introduces a network architecture, called dynoNet, utilizing linear dynamical operators as elementary building blocks. Owing to the dynamical nature of these blocks, dynoNet networks are tailored for sequence modeling and system identification purposes. The back-propagation behavior of the linear dynamical operator with respect to both its parameters and its input sequence is defined. This enables end-to-end training of structured networks containing linear dynamical operators and other differentiable units, exploiting existing deep learning software. Examples show the effectiveness of the proposed approach on well-known system identification benchmarks. K E Y W O R D Smachine learning, neural networks, system identification INTRODUCTION ContributionThis article introduces dynoNet, a neural network architecture tailored for sequence modeling and dynamical system learning (a.k.a. system identification). The network is designed to process time series of arbitrary length and contains causal linear time-invariant (LTI) dynamical operators as building blocks. These LTI layers are parametrized in terms of rational transfer functions, and thus apply infinite impulse response (IIR) filtering to their input sequence. In the dynoNet architecture, the LTI layers are combined with static (i.e., memoryless) nonlinearities which can be either elementary activation functions applied channelwise; fully connected feed-forward neural networks; or other differentiable operators (e.g., polynomials). Both the LTI and the static layers defining a dynoNet are in general multi-input multi-output (MIMO) and can be interconnected in an arbitrary fashion.Overall, the dynoNet architecture can represent rich classes of nonlinear, causal dynamical relations. Moreover, dynoNet networks can be trained end-to-end by plain back-propagation using standard deep learning (DL) software. Technically, this is achieved by introducing the LTI dynamical layer as a differentiable operator, endowed with a well-defined forward and backward behavior and thus compatible with reverse-mode automatic differentiation. 1 Special care is taken to devise closed-form expressions for the forward and backward operations that are convenient from a computational perspective.A software implementation of the linear dynamical operator based on the PyTorch DL framework 2 has been developed and is available in the GitHub repository https://github.com/forgi86/dynonet.git.

show abstract

Section: Representational Power Of Dynonet Architecturesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

dynoNet: A neural network architecture for learning dynamical systems

Forgione

Piga

2021

Adaptive Control & Signal

View full text Add to dashboard Cite

show abstract

“…Inspired by this observation, an alternative approach to the given attitude estimation task is to train a neural network end-to-end on the raw IMU data of a large variety of experimental datasets with ground truth measurements. Considering the success of neural networks in other system identification tasks [18,19], it seems promising to employ them for robust attitude estimation.…”

Section: The Potential Of Neural Network In Inertial Attitude Estimationmentioning

confidence: 99%

“…TCNs are stateless feed-forward neural networks [18], which are able to model dynamic systems by processing windows of a fixed size at once instead of samples sequentially. Transformers are the current state-of-the-art architectures for natural language processing, because of their ability to process relations between two distant points in time [30].…”

Section: Choice Of the Neural Network Structurementioning

confidence: 99%

RIANN—A Robust Neural Network Outperforms Attitude Estimation Filters

Weber

Gühmann

Seel

2021

View full text Add to dashboard Cite

Inertial-sensor-based attitude estimation is a crucial technology in various applications, from human motion tracking to autonomous aerial and ground vehicles. Application scenarios differ in characteristics of the performed motion, presence of disturbances, and environmental conditions. Since state-of-the-art attitude estimators do not generalize well over these characteristics, their parameters must be tuned for the individual motion characteristics and circumstances. We propose RIANN, a ready-to-use, neural network-based, parameter-free, real-time-capable inertial attitude estimator, which generalizes well across different motion dynamics, environments, and sampling rates, without the need for application-specific adaptations. We gather six publicly available datasets of which we exploit two datasets for the method development and the training, and we use four datasets for evaluation of the trained estimator in three different test scenarios with varying practical relevance. Results show that RIANN outperforms state-of-the-art attitude estimation filters in the sense that it generalizes much better across a variety of motions and conditions in different applications, with different sensor hardware and different sampling frequencies. This is true even if the filters are tuned on each individual test dataset, whereas RIANN was trained on completely separate data and has never seen any of these test datasets. RIANN can be applied directly without adaptations or training and is therefore expected to enable plug-and-play solutions in numerous applications, especially when accuracy is crucial but no ground-truth data is available for tuning or when motion and disturbance characteristics are uncertain. We made RIANN publicly available.

show abstract

“…Recently, the connection between nn and nonlinear system identification of black-box model has been highlighted, e.g., [Andersson et al, 2019, 11-19 Dec, Ljung et al, 2020, Schoukens and Ljung, 2019. In Ljung et al [2020] and Andersson et al [2019, 11-19 Dec] feedforward nns are presented as a special case of the nonlinear autoregressive models (narx) where multiple narx models are stacked on top of each other.…”

mentioning

confidence: 99%