RNN- and LSTM-Based Soft Sensors Transferability for an Industrial Process

Abstract

The design and application of Soft Sensors (SSs) in the process industry is a growing research field, which needs to mediate problems of model accuracy with data availability and computational complexity. Black-box machine learning (ML) methods are often used as an efficient tool to implement SSs. Many efforts are, however, required to properly select input variables, model class, model order and the needed hyperparameters. The aim of this work was to investigate the possibility to transfer the knowledge acquired in the design of a SS for a given process to a similar one. This has been approached as a transfer learning problem from a source to a target domain. The implementation of a transfer learning procedure allows to considerably reduce the computational time dedicated to the SS design procedure, leaving out many of the required phases. Two transfer learning methods have been proposed, evaluating their suitability to design SSs based on nonlinear dynamical models. Recurrent neural structures have been used to implement the SSs. In detail, recurrent neural networks and long short-term memory architectures have been compared in regard to their transferability. An industrial case of study has been considered, to evaluate the performance of the proposed procedures and the best compromise between SS performance and computational effort in transferring the model. The problem of labeled data scarcity in the target domain has been also discussed. The obtained results demonstrate the suitability of the proposed transfer learning methods in the design of nonlinear dynamical models for industrial systems.

Transfer Learning in Wastewater Treatment Plant Control Design: From Conventional to Long Short-Term Memory-Based Controllers

“…In that sense, TL techniques can be categorised into three classes as a function of the data availability in the source and target domains or scenarios ([ 21 ] Chapter 4, [ 22 ]): Inductive Transfer Learning: In inductive transfer learning, the source and target domains do not show data scarcity problems. Therefore, the transfer model can be designed and firstly trained in the source domain and then fine-tuned in the target domain in order to adapt its behaviour to its final application.…”

“…TL techniques have been adopted mainly to design and implement soft-sensors in those harsh environments, showing a lack of measurements. In [ 22 ], TL techniques were considered to design a soft-sensor which would be deployed over a sulphur recovery unit. The problem there is that this environment shows a severe problem of data scarcity; therefore, a traditional ANN training process cannot be performed.…”

“…To alleviate this, the authors proposed the adoption of TL to design and implement the soft-sensor in an environment without data scarcity problems (the source domain). Then, the obtained soft-sensor was transferred into the environment with the scarcity problem (the target domain) and fine-tuned to adapt its behaviour to this environment [ 22 ]. In our case, we propose the adoption of the Transfer Learning-based Control design approach to implement and design the complete control strategy of a general-purpose WWTP.…”

A systematic review on application of deep learning in digestive system image processing

“…Each point in the sequence generates an internal signal fed through the neural network to the next layer. Hidden layers preserve information in the observed sequence and updates it in real-time [ 100 ]. Medical reports are typically processed by RNN.…”

Estimation of Mechanical Power Output Employing Deep Learning on Inertial Measurement Data in Roller Ski Skating

“…Though CNN has better data modelling power than DBN, it seems that the typical CNNs are not suitable for time-sequential modelling of data. Recurrent Neural Networks (RNNs) have the advantage over the aforementioned other models (e.g., DBN and CNN), which model time-sequential data obtained from different sources, such as sensors [ 21 , 22 , 23 , 24 , 25 , 26 , 27 ]. However, typical RNNs have the limitation of vanishing gradient problems while modelling long sequences of data.…”