Novel Model Based on Stacked Autoencoders with Sample‐Wise Strategy for Fault Diagnosis

Abstract

Autoencoders are used for fault diagnosis in chemical engineering. To improve their performance, experts have paid close attention to regularized strategies and the creation of new and effective cost functions. However, existing methods are modified on the basis of only one model. This study provides a new perspective for strengthening the fault diagnosis model, which attempts to gain useful information from a model (teacher model) and applies it to a new model (student model). It pretrains the teacher model by fitting ground truth labels and then uses a sample-wise strategy to transfer knowledge from the teacher model. Finally, the knowledge and the ground truth labels are used to train the student model that is identical to the teacher model in terms of structure. The current student model is then used as the teacher of next student model. After step-by-step teacher-student reconfiguration and training, the optimal model is selected for fault diagnosis. Besides, knowledge distillation is applied in training procedures. The proposed method is applied to several benchmarked problems to prove its effectiveness.

Principal Components Versus Autoencoders for Dimensionality Reduction: A Case of Super-Resolved Outputs From Prisma Hyperspectral Mission Data

“…Supervised finetuning of the trained network by backpropagation forms the third phase. Spectral features generated from this pre-training perform better than the traditional information extraction strategies (Kong & Yan, 2019).…”

Quality Assessment of Dimensionality Reduction Techniques on Hyperspectral Data: A Neural Network Based Approach

“…Generally, this problem prevented the training of very deep neural networks and was referred to as the vanishing gradient problem. This problem can be reduced considerably by the process of pre-training (Diehao Kong et al, 2019). Features learned by pre-training a deep autoencoder structure produce spectral features that outperform conventional feature extraction methods.…”

“…Pre-training is based on the assumption that it is easier to train a shallow network instead of a deep network, which also reduces generalization error. Deep neural networks can easily jump out of local minima with the help of pretraining (Diehao Kong et al, 2019).…”