A Deep Learning Approach for Process Data Visualization Using t-Distributed Stochastic Neighbor Embedding

Abstract

A generic process visualization method is introduced, which visualizes real-time process information and correlations among variables on a 2D map using parametric t-SNE. As an unsupervised learning method, it learns the mapping by minimizing the Kullback−Leibler divergence between the original high-dimensional space and the latent space using a deep neural network. In practice, it is observed that the original parametric t-SNE method lacks generalization and struggles to visualize unseen operating conditions correctly. In this work, two steps to improve its generalization capacity are proposed. In the first step, the neural network is trained with additional dummy data, which is generated to mimic the possible unseen conditions. Additionally, the structure of the neural network is reformulated with a new activation function that was designed to improve generalization for process data. The capability of the proposed approach was tested on two case studies, the Tennessee Eastman Process (TEP) and an industrial pyrolysis reactor. The results indicate that the proposed approach outperforms conventional methods in visualization as well as generalization capacity for unseen process conditions.

Manifold Learning and Clustering for Automated Phase Identification and Alignment in Data Driven Modeling of Batch Processes

“…Finally, PCA can be obtained via singular value decomposition and the dimensionality reduction is limited by the linear correlations found in the input space, t-SNE dimensionality reduction results from minimizing the Kullback-Leibler (KL) divergence over all data points, an normally a bi-dimensional or tri-dimensional space is selected as output to allow visualization of embedded data. The use of t-SNE for applications in data driven modeling has being investigated in very recent years, however the focus has been limited to visualization and fault identification (Zhu et al, 2019;Zheng and Zhao, 2020). In this work, t-SNE was chosen because the mentioned characteristics of the method fit well with the requirements of the application for process phase identification.…”

“…This approach has been already tested in different applications, however tuning the ANN to reproduce the manifold learning is rather complex task with many degrees of freedom. Zhu et al (2019) propose an algorithm to implement this approach in the visualization of process data through parametric t-SNE. In this paper an alternative approach is implemented based on SVM for regression.…”

Out-of-sample data visualization using bi-kernel t-SNE

“…In literature, 21 it is mentioned that fine-tuning the trained NN with a small group of unrelated samples can lead to a good result. Zhu et al 26 generated additional dummy data to fine tune a reformulated-structured pt-SNE for outlier mapping to realize good industrial process data visualization. However, how to generate additional unrelated data for fine-tuning is still a great challenge.…”

PCSPred_SC: Prediction of Protein Citrullination Sites Using an Effective Sequence-Based Combined Method

“…By matching distances between high-dimensional and low-dimensional spaces, t-distributed stochastic neighbor embedding (t-SNE) is a dimensionality reduction algorithm retaining the original clustering [46]. The whole procedure of the t-SNE is given in the following steps.…”

“…Thirdly, the effects of the over-sampling methods including random over-sampling (ROS), synthetic minority over-sampling technique (SMOTE) [40], Border-line SMOTE [41], SVM-SMOTE [42] and Adasyn [43] are systematically explored using the top 2 prediction algorithms that achieve the best performance. Finally, to determine the best prediction model, different feature selection methods including mutual information (MI) [44], autoencoder (AE) [45], and t-distributed stochastic neighbor embedding (t-SNE) [46] are respectively incorporated into the top 2 models constructed by a combination of the prediction algorithm and over-sampling methods. Compared with exiting methods, experimental results demonstrate that the proposed method achieves a superior performance in terms of various performance measures.…”