Novel Transformer Based on Gated Convolutional Neural Network for Dynamic Soft Sensor Modeling of Industrial Processes

Geng, Zhiqiang; Chen, Zhiwei; Meng, Qingchao; Han, Yongming

doi:10.1109/tii.2021.3086798

Cited by 103 publications

(27 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, to address the LSTM's inadequacies, a variational autoencoderbased LSTM was designed, which adopts batch training and L2 regularization techniques to learn crucial data information from various process data [10]. In [14], another improved method of LSTM called Gated Convolutional Neural Networkbased Transformer (GCT) was implemented to deal with the gradient vanishing and the parallel computing difficulties. Also, overfitting is a problem that is easily neglected in soft sensor modeling.…”

Section: A Data-driven Soft Sensormentioning

confidence: 99%

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

Ren

Wang

Laili

et al. 2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

Soft sensor, as an important paradigm for industrial intelligence, is widely used in industrial production to achieve efficient monitoring and prediction of production status including product quality. Data-driven soft sensor methods have attracted attention, which still have challenges because of complex industrial data with diverse characteristics, nonlinear relationships and massive unlabeled samples. In this paper, a data-driven self-supervised long short term memory-deep factorization machine (LSTM-DeepFM) model is proposed for industrial soft sensor, in which a framework mainly including pretraining and finetuning stages is proposed to explore diverse industrial data characteristics. In the pretraining stage, LSTM-Autoencoder is first unsupervised pretrained. Then, based on two self-supervised mask strategies, LSTM-Deep can explore the interdependencies between features as well as the dynamic fluctuation in time series. In the finetuning stage, relying on pretrained representation, the temporal, high-dimensional and low-dimensional features can be extracted from the LSTM, Deep and FM components respectively. Finally, experiments on the real-world mining dataset demonstrate that the proposed method achieves state-of-the-art comparing with stacked autoencoder (SAE) based models, variational autoencoder (VAE) based models, and semisupervised parallel DeepFM (SS-PdeepFM), etc.

show abstract

Section: A Data-driven Soft Sensormentioning

confidence: 99%

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

Ren

Wang

Laili

et al. 2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

show abstract

“…Yang [17] et al constructed a model of cement rotary kiln firing system based on BP networks and achieved good fitting results with good generalization ability. Geng [18] et al constructed a dynamic soft measurement model of industrial process based on CNN network, which cleverly solved the influence of high nonlinearity and noise of industrial process data on soft measurement, and concluded that their method has certain superiority through comparison experiments.…”

Section: Related Workmentioning

confidence: 99%

A prediction model for cement specific surface area based on Dual Temporal Extraction Network

Sun

Guo

Liu

et al. 2022

Preprint

View full text Add to dashboard Cite

The accuracy of the predicted value of specific surface area plays a significant guiding function in the production and scheduling of cement, as it is a major index impacting the quality of cement. However, due to data characteristics in the cement grinding process, such as time delay, strong coupling and nonlinearity, and the traditional prediction model being a static small sample feature acquisition method based on time correlation, problems such as poor data feature representation and low accuracy exist. In order to achieve an accurate prediction of cement specific surface area, we propose a deep learning prediction model based on Dual Temporal Extraction Network (DTENet). This network is divided into two parts: encoder and decoder. The network uses a dual temporal feature extraction mechanism in the encoder part. Different from the LSTM network which only focuses on the correlation information of a single time step, we realize the mechanism of temporal feature extraction in the short term and long term respectively by constructing a two-stage sliding window to send the data into different temporal feature extraction networks. This network can greatly improve the prediction accuracy for cement specific surface area data with large time lag, redundancy and variable working conditions. In the decoder part, we use the channel attention approach to enhance the spatial information extraction capability of the model. The result of the experiments shows that our model has superior accuracy in cement specific surface area prediction when compared to LSTM, XGBoost and ARIMA models.

show abstract

“…The development of artificial neural network algorithms has been hot in recent years, and this approach is also widely used in soft sensor modeling. For example, artificial neural network (NN) and support vector regression (SVR), which are used extensively as baseline methods; , deep belief networks (DBN), which build a joint probability distribution between data and labels; , autoencoder networks (AE), which use input data for supervision to guide the network in learning mapping relationships; − ,, long- and short-term memory networks (LSTM), which can “remember” and can be applied to time series; ,− and convolutional neural networks (CNN), which is based on visual principles and pays more attention to local features. − For soft sensor modeling, neural networks extract useful features from many easily accessible auxiliary variables and then build a model between the key variables and the extracted features for prediction.…”

Section: Introductionmentioning

confidence: 99%

Horizontal Data Augmentation Strategy for Industrial Quality Prediction

et al. 2022

View full text Add to dashboard Cite

In recent years, neural network-based soft sensor technology has been widely used in industrial production processes and has excellent optimization, monitoring, and quality prediction performance. This paper proposes a horizontal data augmentation strategy to provide highly available data for subsequent prediction models, called the combined autoencoder data augmentation (CADA) strategy. This paper has developed a CADA-based convolutional neural network (CADA-CNN) soft sensor model and applied it to the process of industrial debutanizer and industrial steam volume. In terms of method validation, this paper compares the output data of the proposed CADA by the Spearman correlation coefficient to verify the strategy’s feasibility. Then, the output data of the CADA strategy is fed into the artificial neural network (NN), support vector regression (SVR), and convolutional neural network (CNN) for comparison experiments. The final experimental results show that our proposed CADA-CNN model has lower prediction error and better prediction error distribution.

show abstract

Novel Transformer Based on Gated Convolutional Neural Network for Dynamic Soft Sensor Modeling of Industrial Processes

Cited by 103 publications

References 21 publications

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

A prediction model for cement specific surface area based on Dual Temporal Extraction Network

Horizontal Data Augmentation Strategy for Industrial Quality Prediction

Contact Info

Product

Resources

About