A Data-Driven Self-Supervised LSTM-DeepFM Model for Industrial Soft Sensor

Abstract

Soft sensor, as an important paradigm for industrial intelligence, is widely used in industrial production to achieve efficient monitoring and prediction of production status including product quality. Data-driven soft sensor methods have attracted attention, which still have challenges because of complex industrial data with diverse characteristics, nonlinear relationships and massive unlabeled samples. In this paper, a data-driven self-supervised long short term memory-deep factorization machine (LSTM-DeepFM) model is proposed for industrial soft sensor, in which a framework mainly including pretraining and finetuning stages is proposed to explore diverse industrial data characteristics. In the pretraining stage, LSTM-Autoencoder is first unsupervised pretrained. Then, based on two self-supervised mask strategies, LSTM-Deep can explore the interdependencies between features as well as the dynamic fluctuation in time series. In the finetuning stage, relying on pretrained representation, the temporal, high-dimensional and low-dimensional features can be extracted from the LSTM, Deep and FM components respectively. Finally, experiments on the real-world mining dataset demonstrate that the proposed method achieves state-of-the-art comparing with stacked autoencoder (SAE) based models, variational autoencoder (VAE) based models, and semisupervised parallel DeepFM (SS-PdeepFM), etc.

An Intelligent Thermal Compensation System Using Edge Computing for Machine Tools

“…Zhou et al [ 13 ] proposed an intelligent anomaly detection variational long-short-term memory (VLSTM) learning model based on reconstructed feature representation, which can effectively address imbalanced and high-dimensional problems in industrial big data and significantly improve the accuracy of data anomaly monitoring while reducing the error rate in the industry. Ren et al [ 14 ] proposed a data-driven self-supervised long- and short-term memory deep factorization machine (LSTM-DeepFM) model driven by data for soft industrial measurement, including a framework of pre-training and fine-tuning stages to explore different features of industrial data. Mateus et al [ 15 ] used an LSTM model to predict the future condition of industrial paper machine equipment based on sensor data, maximizing industrial plant maintenance and supporting decision-making about equipment availability.…”

Time-aware sport goods sale prediction for healthcare with privacy-preservation

“…In addition, data sparsity is inevitable in the big data environment [39] , [40] , [41] . To tackle this issue, many researchers have proposed various resolutions based on machine learning, e.g., Graph Neural Network [42] , [43] , Attention Mechanism [44] , Approximate Nearest Neighbor search [45] , Deep Correlation Mining [46] and LSTM [47] . Due to the page limit of the paper, we will not introduce their details one by one here.…”

Improving resilience of sensors in planetary exploration using data-driven models

“…Input gate (i t ) selects the input value that needs to update the current memory state. It selects through combining output of previous cell (h t−1 ) and current input (x t ) and applying sigmoid activation function (σ) as given in equation (12). Then, tangent hyperbolic function (tanh) is applied on similar combination of parameters to generate candidate cell state vector ( Ct ).…”