Convolutional neural networks combined with Runge–Kutta methods

Zhu, Mai; Chang, Bo; Fu, Chong

doi:10.1007/s00521-022-07785-2

Cited by 18 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For continuous function f (v t ) with K 1 , the Euler method shown as v t+∆t has high truncation error because it is a firstorder approximation to the true solution. This introduces the drawback of the first-order Euler method to the unitary transformation module, which is difficult to learn the long-range information in the continuous network, reducing accuracy of model outputs (Zhu, Chang, and Fu 2022;Li et al 2021a). Therefore, due to the limitation of Euler method, it results in the low training efficiency of QINNs.…”

Section: Unitary Transformation Of Qinns Are An Eular Methodsmentioning

confidence: 99%

Quantum-Inspired Neural Network with Runge-Kutta Method

Fan,

Zhang,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

In recent years, researchers have developed novel Quantum-Inspired Neural Network (QINN) frameworks for the Natural Language Processing (NLP) tasks, inspired by the theoretical investigations of quantum cognition. However, we have found that the training efficiency of QINNs is significantly lower than that of classical networks. We analyze the unitary transformation modules of existing QINNs based on the time displacement symmetry of quantum mechanics and discover that they are resembling a mathematical form similar to the first-order Euler method. The high truncation error associated with Euler method affects the training efficiency of QINNs. In order to enhance the training efficiency of QINNs, we generalize QINNs' unitary transformation modules to the Quantum-like high-order Runge-Kutta methods (QRKs). Moreover, we present the results of experiments on conversation emotion recognition and text classification tasks to validate the effectiveness of the proposed approach.

show abstract

Section: Unitary Transformation Of Qinns Are An Eular Methodsmentioning

confidence: 99%

Quantum-Inspired Neural Network with Runge-Kutta Method

Fan,

Zhang,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Describing the evolution of dynamical systems (or sub-systems) accurately is an omnipresent task in scientific computing. The time integration schemes of differential operators have been correspondingly considered in different fields (e.g., pairing Runge-Kutta integration schemes with ANNs 55,159,184,185 ). For the time integration of multiscale physical systems, Liu et al 159 suggested a hierarchical time-steppers (HiTSs) framework.…”

Section: Data-driven Surrogate Sub-modelingmentioning

confidence: 99%

Review: Machine learning for advancing low-temperature plasma modeling and simulation

Trieschmann,

Vialetto,

Gergs

2023

J. Micro/Nanopattern. Mats. Metro.

View full text Add to dashboard Cite

Machine learning has had an enormous impact in many scientific disciplines. It has also attracted significant interest in the field of low-temperature plasma (LTP) modeling and simulation in past years. Its application should be carefully assessed in general, but many aspects of plasma modeling and simulation have benefited substantially from recent developments within the field of machine learning and datadriven modeling. In this survey, we approach two main objectives: (a) we review the state-of-the-art, focusing on approaches to LTP modeling and simulation. By dividing our survey into plasma physics, plasma chemistry, plasma-surface interactions, and plasma process control, we aim to extensively discuss relevant examples from literature. (b) We provide a perspective of potential advances to plasma science and technology. We specifically elaborate on advances possibly enabled by adaptation from other scientific disciplines. We argue that not only the known unknowns but also unknown unknowns may be discovered due to the inherent propensity of data-driven methods to spotlight hidden patterns in data.

show abstract

“…Therefore, it is necessary to balance the relationship between accuracy and computational complexity when choosing the Runge-Kutta strategy. However, applying fourth-order or higher-order Runge-Kutta strategies may lead to excessive computational complexity, which could affect the training speed and practicality of the model [28,29]. Therefore, in this paper, we will not discuss the application of higher-order methods for the time being.…”

Section: Pre-activation Runge-kutta Residual Blockmentioning

confidence: 99%

A rolling bearing fault diagnosis method based on Markov transition field and multi-scale Runge-Kutta residual network

Ding,

Rui,

Lei

et al. 2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

In order to address the problem that one- dimensional convolutional neural networks is difficult to extract the local correlation information and mine multi-scale information of rolling bearing fault signals under variable working conditions, a novel fault diagnosis method for rolling bearings based on Markov transition field (MTF) and multi-scale Runge–Kutta residual attention network (MRKRA-Net) is proposed in this paper. Firstly, the original signal is encoded into a two-dimensional image using the MTF method. Then, a multi-scale network is constructed using pre-activation Runge–Kutta residual blocks to extract multi-level features. Secondly, a feature-guided attention mechanism is designed and embedded into the network model to enhance its generalization ability. Finally, the MRKRA-Net model is validated on two different bearing datasets, and the results show that compared with other popular intelligent fault diagnosis methods, MRKRA-Net has higher fault diagnosis accuracy and stronger robustness under both given and variable working conditions.

show abstract

Convolutional neural networks combined with Runge–Kutta methods

Cited by 18 publications

References 13 publications

Quantum-Inspired Neural Network with Runge-Kutta Method

Quantum-Inspired Neural Network with Runge-Kutta Method

Review: Machine learning for advancing low-temperature plasma modeling and simulation

A rolling bearing fault diagnosis method based on Markov transition field and multi-scale Runge-Kutta residual network

Contact Info

Product

Resources

About