Attention-embedded Quadratic Network (Qttention) for Effective and Interpretable Bearing Fault Diagnosis

Liao, Jing-Xiao; Dong, Hang-Cheng; Sun, Zhiqi; Sun, Jing; Zhang, Shiping; Fan, Fenglei

doi:10.48550/arxiv.2206.00390

Cited by 3 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, we conduct experiments using our collected dataset [22]. We carry out a bearing fault test in MIIT Key Laboratory of Aerospace Bearing Technology and Equipment, Harbin Institute of Technology.…”

Section: Case 2: Manually Injected Fault Signals Under Noisy Conditio...mentioning

confidence: 99%

See 1 more Smart Citation

Multi-task neural network blind deconvolution and its application to bearing fault feature extraction

Liao

Dong

Luo

et al. 2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

Blind deconvolution (BD) is one of the effective methods that extract fault-related characteristics in vibration signals. Currently, most BD methods specify an optimization criterion and use frequency or time domain signal independently to optimize a deconvolution filter. However, they prone to overfitting due to the various noises. The time-domain-based BD methods tend to extract fault-unrelated single peak impulse, and the frequency-domain-based BD methods tend to retain the maximum energy frequency component but lose harmonics frequency components. To solve the above issue, we propose a hybrid criterion that combines the kurtosis in time domain and the G-l1/l2 norm in the frequency domain. Two criteria have opposite monotonicity that mutually constrain to avoid overfitting. Moreover, we construct a multi-task 1D convolutional neural network with two branches to achieve an optimal solution. This network realizes the simultaneous optimization of two domains. Experimental results show that our proposed method outperforms other state-of-the-art BD methods.

show abstract

Section: Case 2: Manually Injected Fault Signals Under Noisy Conditio...mentioning

confidence: 99%

“…At last, the BD methods had a wide range of applications. They can act as a stand-alone feature extraction method for fault signals denoising and interpretation or serve as a pre-processing tool to assist in downstream faults diagnosis tasks [20][21][22][23][24].…”

Section: Introductionmentioning

confidence: 99%

Multi-task neural network blind deconvolution and its application to bearing fault feature extraction

Liao

Dong

Luo

et al. 2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Xu et al built a quadratic network library (QuadraLib) featuring algorithm acceleration to facilitate the application studies of quadratic networks [27]. Liao et al applied the quadratic network to the bearing fault diagnosis on one-dimensional vibration signals, and discovered the attention mechanism inherent in a quadratic neuron (qttention) [11]. Chrysos et al studied the architectures of polynomial networks and achieved the state-of-the-art performance on image classification and generation [33].…”

Section: Related Workmentioning

confidence: 99%

“…This algorithm initializes the quadratic network as the conventional network and controls the learning rate of quadratic terms. Subsequently, quadratic networks were extensively applied to solving real-world problems, e.g., low-dose CT denoising [10] and bearing fault diagnosis [11]. This series of quadratic neuron studies aim to develop deep learning based on new neurons and validate the potential of introducing neuronal diversity into deep learning.…”

Section: Introductionmentioning

confidence: 99%

A Review of Artificial Intelligence for Games

Fan¹,

Wu²,

Tian³

2020

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Inspired by neuronal diversity in the biological neural system, a plethora of studies proposed to design novel types of artificial neurons and introduce neuronal diversity into artificial neural networks. Recently proposed quadratic neuron, which replaces the inner-product operation in conventional neurons with a quadratic one, have achieved great success in many essential tasks. Despite the promising results of quadratic neurons, there is still an unresolved issue: Is the superior performance of quadratic networks simply due to the increased parameters or due to the intrinsic expressive capability? Without clarifying this issue, the performance of quadratic networks is always suspicious. Additionally, resolving this issue is reduced to finding killer applications of quadratic networks. In this paper, with theoretical and empirical studies, we show that quadratic networks enjoy parametric efficiency, thereby confirming that the superior performance of quadratic networks is due to the intrinsic expressive capability. This intrinsic expressive ability comes from that quadratic neurons can easily represent nonlinear interaction, while it is hard for conventional neurons. Theoretically, we derive the approximation efficiency of the quadratic network over conventional ones in terms of real space and manifolds. Moreover, from the perspective of the Barron space, we demonstrate that there exists a functional space whose functions can be approximated by quadratic networks in a dimension-free error, but the approximation error of conventional networks is dependent on dimensions. Empirically, experimental results on synthetic data, classic benchmarks, and real-world applications show that quadratic models broadly enjoy parametric efficiency, and the gain of efficiency depends on the task. We have shared our code in https://github.com/asdvfghg/quadratic efficiency.

show abstract

Multi-task Neural Network Blind Deconvolution and its Application to Bearing Fault Feature Extraction

Liao¹,

Dong²,

Luo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Blind deconvolution (BD) is one of the effective methods that help pre-process vibration signals and assist in bearing fault diagnosis. Currently, most BD methods design an optimization criterion and use frequency or time domain information independently to optimize a deconvolution filter. It recovers weak periodic impulses related to incipient faults. However, the random noise interference may cause the optimizer to overfit. The time-domain-based BD methods tend to extract fault-unrelated single peak impulse, and the frequency-domain-based BD methods tend to retain the maximum energy frequency component, which will lose the fault-related harmonics frequency components. To solve the above issue, we propose a hybrid criterion that combines the kurtosis for time domain optimization and the $G-l_1/l_2$ norm for the frequency domain. These two criteria are monotonically increasing and decreasing, so they mutually constrain to avoid overfitting. After that, we design a multi-task one-dimensional convolutional neural network with time and frequency branches to achieve an optimal solution for this hybrid criterion. The multi-task neural network realizes the simultaneous optimization of two domains. Experimental results show that our proposed method outperforms other state-of-the-art methods.

show abstract

Attention-embedded Quadratic Network (Qttention) for Effective and Interpretable Bearing Fault Diagnosis

Cited by 3 publications

References 25 publications

Multi-task neural network blind deconvolution and its application to bearing fault feature extraction

Multi-task neural network blind deconvolution and its application to bearing fault feature extraction

A Review of Artificial Intelligence for Games

Multi-task Neural Network Blind Deconvolution and its Application to Bearing Fault Feature Extraction

Contact Info

Product

Resources

About