A Memristive Neural Networks Described by Differential-Algebraic Systems

Chen, Boshan; Chen, Jiejie; Zeng, Zhigang

doi:10.1109/icist.2018.8426153

Cited by 12 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The learned policies were then tested online in a closed loop within the training distribution and in drastically distinct settings. This experimental protocol allows for the principled assessment of performance and generalization capabilities of liquid networks compared with modern deep models (38,41,42).…”

Section: Fly-to-target Task Trainingmentioning

confidence: 99%

“…5B) (50). These brain-inspired models are instances of continuous-time (CT) neural networks (35,41) that can be trained via gradient descent in modern automatic differentiation frameworks. Liquid networks exhibit stable and bounded behavior, yield superior expressivity within the family of CT neural models (35,41), and give rise to improved performance on a wide range of time series prediction tasks compared with advanced, recurrent neural network models (50).…”

Section: Liquid Network: Brain-inspired Neural Modelsmentioning

confidence: 99%

“…These brain-inspired models are instances of continuous-time (CT) neural networks (35,41) that can be trained via gradient descent in modern automatic differentiation frameworks. Liquid networks exhibit stable and bounded behavior, yield superior expressivity within the family of CT neural models (35,41), and give rise to improved performance on a wide range of time series prediction tasks compared with advanced, recurrent neural network models (50). In particular, a sparse network configuration composed of fewer than two dozen LTC neurons supplied with convolutional heads showed great promise in learning, end to end, to map high-dimensional visual input stream of pixels to robust control decisions (18).…”

Section: Liquid Network: Brain-inspired Neural Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Robust flight navigation out of distribution with liquid neural networks

et al. 2023

View full text Add to dashboard Cite

Autonomous robots can learn to perform visual navigation tasks from offline human demonstrations and generalize well to online and unseen scenarios within the same environment they have been trained on. It is challenging for these agents to take a step further and robustly generalize to new environments with drastic scenery changes that they have never encountered. Here, we present a method to create robust flight navigation agents that successfully perform vision-based fly-to-target tasks beyond their training environment under drastic distribution shifts. To this end, we designed an imitation learning framework using liquid neural networks, a brain-inspired class of continuous-time neural models that are causal and adapt to changing conditions. We observed that liquid agents learn to distill the task they are given from visual inputs and drop irrelevant features. Thus, their learned navigation skills transferred to new environments. When compared with several other state-of-the-art deep agents, experiments showed that this level of robustness in decision-making is exclusive to liquid networks, both in their differential equation and closed-form representations.

show abstract

Section: Fly-to-target Task Trainingmentioning

confidence: 99%

Section: Liquid Network: Brain-inspired Neural Modelsmentioning

confidence: 99%

Section: Liquid Network: Brain-inspired Neural Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Robust flight navigation out of distribution with liquid neural networks

et al. 2023

View full text Add to dashboard Cite

show abstract

“…These tools provide differentiable functions for physical simulations, which enable close integration with deep learning frameworks by leveraging their automatic differentiation functionality. Hybrid approaches that combine machine learning techniques with numerical PDE solvers (Wang et al, 2020;Illarramendi et al, 2022), have attracted a significant amount of interest due to their capabilities for generalization (Chen et al, 2018). In this context, neural networks are typically used to model or replace a part of the conventional PDE solver to improve aspects of the solving process.…”

Section: Neural Network With Differentiable Pde Solversmentioning

confidence: 99%

Incomplete to complete multiphysics forecasting: a hybrid approach for learning unknown phenomena

Tathawadekar,

Doan,

Silva

et al. 2023

DCE

View full text Add to dashboard Cite

Modeling complex dynamical systems with only partial knowledge of their physical mechanisms is a crucial problem across all scientific and engineering disciplines. Purely data-driven approaches, which only make use of an artificial neural network and data, often fail to accurately simulate the evolution of the system dynamics over a sufficiently long time and in a physically consistent manner. Therefore, we propose a hybrid approach that uses a neural network model in combination with an incomplete partial differential equations (PDEs) solver that provides known, but incomplete physical information. In this study, we demonstrate that the results obtained from the incomplete PDEs can be efficiently corrected at every time step by the proposed hybrid neural network—PDE solver model, so that the effect of the unknown physics present in the system is correctly accounted for. For validation purposes, the obtained simulations of the hybrid model are successfully compared against results coming from the complete set of PDEs describing the full physics of the considered system. We demonstrate the validity of the proposed approach on a reactive flow, an archetypal multi-physics system that combines fluid mechanics and chemistry, the latter being the physics considered unknown. Experiments are made on planar and Bunsen-type flames at various operating conditions. The hybrid neural network—PDE approach correctly models the flame evolution of the cases under study for significantly long time windows, yields improved generalization and allows for larger simulation time steps.

show abstract

“…For example, a framework useful for automatic learning PDEs from data [24,25] has been proposed. Another group used an adjoint method to learn differential equations parameterized with neural networks [26], while Ayed et al [27] proposed a framework for learning models using a purely data-driven approach in partially observable settings.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous data assimilation and cardiac electrophysiology model correction using differentiable physics and deep learning

Kashtanova,

Pop,

Ayed

et al. 2023

Interface Focus.

View full text Add to dashboard Cite

Modelling complex systems, like the human heart, has made great progress over the last decades. Patient-specific models, called ‘digital twins’, can aid in diagnosing arrhythmias and personalizing treatments. However, building highly accurate predictive heart models requires a delicate balance between mathematical complexity, parameterization from measurements and validation of predictions. Cardiac electrophysiology (EP) models range from complex biophysical models to simplified phenomenological models. Complex models are accurate but computationally intensive and challenging to parameterize, while simplified models are computationally efficient but less realistic. In this paper, we propose a hybrid approach by leveraging deep learning to complete a simplified cardiac model from data. Our novel framework has two components, decomposing the dynamics into a physics based and a data-driven term. This construction allows our framework to learn from data of different complexity, while simultaneously estimating model parameters. First, using in silico data, we demonstrate that this framework can reproduce the complex dynamics of cardiac transmembrane potential even in the presence of noise in the data. Second, using ex vivo optical data of action potentials (APs), we demonstrate that our framework can identify key physical parameters for anatomical zones with different electrical properties, as well as to reproduce the AP wave characteristics obtained from various pacing locations. Our physics-based data-driven approach may improve cardiac EP modelling by providing a robust biophysical tool for predictions.

show abstract

A Memristive Neural Networks Described by Differential-Algebraic Systems

Cited by 12 publications

References 18 publications

Robust flight navigation out of distribution with liquid neural networks

Robust flight navigation out of distribution with liquid neural networks

Incomplete to complete multiphysics forecasting: a hybrid approach for learning unknown phenomena

Simultaneous data assimilation and cardiac electrophysiology model correction using differentiable physics and deep learning

Contact Info

Product

Resources

About