System identification through Lipschitz regularized deep neural networks

Negrini, Elisa; Citti, Giovanna; Capogna, Luca

doi:10.1016/j.jcp.2021.110549

Cited by 13 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-Loss function: in our tests, the training effect of the L 1 loss is slightly better than that of L 2 ; -Regularizer: Although strict regularization may lead to the decline in end-to-end accuracy, in our experiments, the Lipschitz regularizer [28] slightly improves the estimation of physical parameters; -Training samples: the extreme values of the Coriolis force and centrifugal force of the manipulator only appear in a particular status, and the distribution of the sample must be specifically considered for this.…”

Section: Methodsmentioning

confidence: 72%

Bidirectional Dynamic Neural Networks with Physical Analyzability

Zhao

Lan

et al. 2022

Preprint

View full text Add to dashboard Cite

The rapid growth in research exploiting deep learning to predict mechanical systems has revealed a new route for system identification; however, the analytic model as a white box has not been replaced in applications because of its open physical information. In contrast, the models generated by end-to-end learning usually lack the ability of physical analysis, which makes them inapplicable in many situations. Consequently, high-accuracy modeling with physical analyzability becomes a necessity. In this paper, we introduce bidirectional dynamic neural networks, a deep learning framework that can infer the dynamics of physical systems from control signals and observed state trajectories. Based on forward dynamics, we train the neural ordinary differential equations in a trajectory backtracking algorithm. With the trained model, the inverse dynamics can be calculated and based on Lagrangian Mechanics, the physical parameters of the mechanical system can be estimated, including inertia, Coriolis and centrifugal forces, and gravity. As a result, the model can seamlessly incorporate prior knowledge, learn unknown dynamics without human intervention, and provide information as transparent as analytic models. We demonstrate our method on simulated 2-axis and 6-axis robots to evaluate model accuracy, including physical parameters and verified its applicability on real 7-axis robots. The experimental results show that this method is superior to the existing methods. This framework provides a new idea for system identification by providing interpretable, physically consistent models for physical systems.

show abstract

Section: Methodsmentioning

confidence: 72%

Bidirectional Dynamic Neural Networks with Physical Analyzability

Zhao

Lan

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Applying machine learning techniques to dynamical systems have become a recent trend, now that there is an abundance of data. Some researchers have taken to performing system identification by identifying the governing symbolic equations through data [2,19,4], while others have taken to representing the governing equations through neural networks [23,18,5,17,16]. Delay-embeddings in terms of system identification have also been examined in [12,5].…”

Section: Related Workmentioning

confidence: 99%

Parameter Inference of Time Series by Delay Embeddings and Learning Differentiable Operators

Lin¹,

Eckhardt²,

Martin³

et al. 2022

Preprint

View full text Add to dashboard Cite

A common issue in dealing with real-world dynamical systems is identifying system parameters responsible for its behavior. A frequent scenario is that one has time series data, along with corresponding parameter labels, but there exists new time series with unknown parameter labels, which one seeks to identify. We tackle this problem by first delay-embedding the time series into a higher dimension to obtain a proper ordinary differential equation (ODE), and then having a neural network learn to predict future time-steps of the trajectory given the present time-step. We then use the learned neural network to backpropagate prediction errors through the parameter inputs of the neural network in order to obtain a gradient in parameter space. Using this gradient, we can approximately identify parameters of time series. We demonstrate the viability of our approach on the chaotic Lorenz system, as well as real-world data with the Hall-effect Thruster (HET).

show abstract

“…These observations can be viewed as a 3dimensional time series that estimate the empirical covariance Γ obs as in [11]. The inverse problem involves learning the parameter u given these observations, also known as parameter identification [41]. Following [11], we endow a log-Normal prior on u: log u ∼ N (µ 0 , σ 2 0 ) with µ 0 = (2.0, 1.2, 3.3) and σ 0 = (0.2, 0.5, 0.15).…”

Section: Lorenz Systemmentioning

confidence: 99%

Bayesian Spatiotemporal Modeling for Inverse Problems

Lan¹,

Li²,

Pasha³

2022

Preprint

View full text Add to dashboard Cite

Inverse problems with spatiotemporal observations are ubiquitous in scientific studies and engineering applications. In these spatiotemporal inverse problems, observed multivariate time series are used to infer parameters of physical or biological interests. Traditional solutions for these problems often ignore the spatial or temporal correlations in the data (static model), or simply model the data summarized over time (time-averaged model). In either case, the data information that contains the spatiotemporal interactions is not fully utilized for parameter learning, which leads to insufficient modeling in these problems. In this paper, we apply Bayesian models based on spatiotemporal Gaussian processes (STGP) to the inverse problems with spatiotemporal data and show that the spatial and temporal information provides more effective parameter estimation and uncertainty quantification (UQ). We demonstrate the merit of Bayesian spatiotemporal modeling for inverse problems compared with traditional static and time-averaged approaches using a time-dependent advection-diffusion partial different equation (PDE) and three chaotic ordinary differential equations (ODE). We also provide theoretic justification for the superiority of spatiotemporal modeling to fit the trajectories even it appears cumbersome (e.g. for chaotic dynamics).

show abstract

System identification through Lipschitz regularized deep neural networks

Cited by 13 publications

References 27 publications

Bidirectional Dynamic Neural Networks with Physical Analyzability

Bidirectional Dynamic Neural Networks with Physical Analyzability

Parameter Inference of Time Series by Delay Embeddings and Learning Differentiable Operators

Bayesian Spatiotemporal Modeling for Inverse Problems

Contact Info

Product

Resources

About