Variational Autoencoder Bidirectional Long and Short-Term Memory Neural Network Soft-Sensor Model Based on Batch Training Strategy

Xie, Wei; Wang, Jiesheng; Cheng, Xing; Guo, Shasha; Zhu, Lei

doi:10.1109/tii.2020.3025204

Cited by 59 publications

(25 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, a deep probabilistic transfer learning framework [12] and a model-agnostic meta-learning method [13] were proposed to improve the model performance when migrating from the relevant source domain to the target domain. In addition, to address the LSTM's inadequacies, a variational autoencoderbased LSTM was designed, which adopts batch training and L2 regularization techniques to learn crucial data information from various process data [10]. In [14], another improved method of LSTM called Gated Convolutional Neural Networkbased Transformer (GCT) was implemented to deal with the gradient vanishing and the parallel computing difficulties.…”

Section: A Data-driven Soft Sensormentioning

confidence: 99%

“…Thus, artificial samples can be obtained by randomly sampling from the distribution. In addition, to cope with the complex data characteristics, an improved LSTM with the variational autoencoder [10] is designed to extract the complex temporal information in the industrial time series, while SS-PdeepFM [11] is used to extract low-dimensional and high-dimensional features in a single time step. In summary, there are still several major challenges with existing data-driven approaches, as follows:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

Ren

Wang

Laili

et al. 2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

Soft sensor, as an important paradigm for industrial intelligence, is widely used in industrial production to achieve efficient monitoring and prediction of production status including product quality. Data-driven soft sensor methods have attracted attention, which still have challenges because of complex industrial data with diverse characteristics, nonlinear relationships and massive unlabeled samples. In this paper, a data-driven self-supervised long short term memory-deep factorization machine (LSTM-DeepFM) model is proposed for industrial soft sensor, in which a framework mainly including pretraining and finetuning stages is proposed to explore diverse industrial data characteristics. In the pretraining stage, LSTM-Autoencoder is first unsupervised pretrained. Then, based on two self-supervised mask strategies, LSTM-Deep can explore the interdependencies between features as well as the dynamic fluctuation in time series. In the finetuning stage, relying on pretrained representation, the temporal, high-dimensional and low-dimensional features can be extracted from the LSTM, Deep and FM components respectively. Finally, experiments on the real-world mining dataset demonstrate that the proposed method achieves state-of-the-art comparing with stacked autoencoder (SAE) based models, variational autoencoder (VAE) based models, and semisupervised parallel DeepFM (SS-PdeepFM), etc.

show abstract

Section: A Data-driven Soft Sensormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

Ren

Wang

Laili

et al. 2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

show abstract

“…The Bi-LSTM model behaves the same with all inputs. The mathematical formulations of Bi-LSTM are presented in detail in [36].…”

Section: Bidirectional Long Short-term Memory (Bi-lstm)mentioning

confidence: 99%

“…In the structure of the traditional RNN and the LSTM model, the propagation of information happens in a forward path, in which case the time t depends only on the information before the time t. In the Bi-LSTM network, unlike in traditional LSTM, flowing the information from the backward layer to the forward layer and upside down is performed by employing a hidden state [36]. Additionally, the advantage of Bi-LSTM over convolutional neural networks (CNNs) is its dependency on the sequence of inputs by taking the forward and backward paths into account.…”

Section: Case Studymentioning

confidence: 99%

Deep Learning-Assisted Short-Term Load Forecasting for Sustainable Management of Energy in Microgrid

et al. 2021

View full text Add to dashboard Cite

Nowadays, supplying demand load and maintaining sustainable energy are important issues that have created many challenges in power systems. In these types of problems, short-term load forecasting has been proposed as one of the management and energy supply modes in power systems. In this paper, after reviewing various load forecasting techniques, a deep learning method called bidirectional long short-term memory (Bi-LSTM) is presented for short-term load forecasting in a microgrid. By collecting relevant features available in the input data at the training stage, it is shown that the proposed procedure enjoys important properties, such as its great ability to process time series data. A microgrid in rural Sub-Saharan Africa, including household and commercial loads, was selected as the case study. The parameters affecting the formation of household and commercial load profiles are considered as input variables, and the total household and commercial load profiles of the microgrid are considered as the target. The Bi-LSTM network is trained by input variables to forecast the microgrid load on an hourly basis by recognizing the consumption pattern. Various performance evaluation indicators such as the correlation coefficient (R), mean squared error (MSE), and root mean squared error (RMSE) are utilized to analyze the forecast results. In addition, in a comparative approach, the performance of the proposed method is compared and evaluated with other methods used in similar studies. The results presented for the training phase show an accuracy of R = 99.81% for the Bi-LSTM network. The test and load forecasting stage are performed by the Bi-STLM network, with an accuracy of R = 99.34% and forecasting errors of MSE = 0.1042 and RMSE = 0.3243. The results confirm the high performance of the proposed Bi-LSTM technique, with a high correlation coefficient when compared to other methods used for short-term load forecasting.

show abstract

“…This can result in a delay in the quality control system as there is no available quality indicator at this time. In the absence of an economical or effective online measurement, soft sensors could serve as an alternative solution [8][9][10][11][12][13][14]. Additionally, with the wide availability of process data in PP processes, increasing data-driven soft sensors have been adopted to predict the MI.…”

Section: Introductionmentioning

confidence: 99%

Soft Sensor Development Based on Quality‐Relevant Slow Feature Analysis and Bayesian Regression with Application to Propylene Polymerization

et al. 2021

View full text Add to dashboard Cite

Data-driven soft sensors are widely used to predict quality indices in propylene polymerization processes to improve the availability of measurements and efficiency. To deal with the nonlinearity and dynamics in propylene polymerization processes, a novel soft sensor based on quality-relevant slow feature analysis and Bayesian regression is proposed in this paper. The proposed method can handle the dynamics of the process better by extracting quality-relevant slow features, which present both the slowly varying characteristic and the correlations with quality indices. Meanwhile, a Bayesian inference model is developed to predict the quality indices, which takes advantages of a probability framework with iterative maximum likelihood techniques for parameter estimation and a sparse constraint for avoiding overfitting. Finally, a case study is conducted with data sampled from a practical industrial propylene polymerization process to demonstrate the effectiveness and superiority of the proposed method.

show abstract

Variational Autoencoder Bidirectional Long and Short-Term Memory Neural Network Soft-Sensor Model Based on Batch Training Strategy

Cited by 59 publications

References 12 publications

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

Deep Learning-Assisted Short-Term Load Forecasting for Sustainable Management of Energy in Microgrid

Soft Sensor Development Based on Quality‐Relevant Slow Feature Analysis and Bayesian Regression with Application to Propylene Polymerization

Contact Info

Product

Resources

About