Modeling and Control of Nonlinear Processes Using Sparse Identification: Using Dropout to Handle Noisy Data

Abstract

Sparse identification of nonlinear dynamics (SINDy) is a recent nonlinear modeling technique that has demonstrated superior performance in modeling complex time-series data in the form of first-order ordinary differential equations (ODEs), which are explicit and continuous in time. However, a crucial step in the SINDy algorithm involves estimating the time derivative of the states from the discrete, measured data. Therefore, the presence of noise can greatly deteriorate the performance if it is not carefully considered and accounted for. In this work, SINDy is used with ensemble learning, where multiple models are identified to improve the overall/final nonlinear model’s performance. Specifically, in the SINDy algorithm, a fraction of the library functions considered for the ODE model representation are randomly dropped out in each submodel to favor model sparsity and stability at the possible risk of lowering the model accuracy. This trade-off is controlled by manipulating the fraction of the library functions dropped out and the total number of models generated, both of which are considered as hyperparameters to be tuned in the proposed algorithm. Data from open-loop simulations of a large-scale chemical plant are generated using the well-known high-fidelity process simulator, Aspen Plus Dynamics, and corrupted with substantial sensor noise to be implemented in the newly proposed algorithm, dropout-SINDy. The dropout-SINDy models obtained from training with the noisy data are then tested in open-loop simulations to demonstrate accurate identification of the steady-state and reasonably close transient behavior under a variety of initial conditions and manipulated input values. Finally, the constructed models are used in a Lyapunov-based model predictive controller to control the large-scale Aspen process, meeting desired closed-loop stability and performance specifications.

Machine Learning-Based Adaptive Regression to Identify Nonlinear Dynamics of Biochemical Systems: A Case Study on Bio 2,3-Butanediol Distillation Process

“…From this perspective, sparsity can be achieved through subset selection from the library using feature selection techniques in machine learning. This approach serves the purpose of identifying relevant functions. , To this end, in this study, we employ SFS as a sparsity-promoting method to identify the essential functions by iteratively selecting them from a nonlinear library that make the greatest contribution to the user-defined evaluation metric such as root-mean-squared error (RMSE) and coefficient of determination ( R 2 ). It starts with an empty subset and iteratively adds one feature at a time until a user-defined stopping criterion is met .…”

“…Traditionally, first-principles models are employed in many fields, such as biochemical science and manufacturing systems, to represent the nonlinear dynamics in the system. In real-world industrial processes, complex nonlinear behaviors and intricate input–output interactions cannot be captured by limited first-principles knowledge of the systems, leading to plant-model mismatch. − The rapid increase in computational power and advancement in machine learning have paved the way for data-driven modeling approaches to emerge as a viable alternative to traditional first-principles models. The primary advantage of adopting data-driven approaches lies in the structural flexibility in modeling complex systems.…”

Modeling of Harmful Algal Bloom Dynamics and Integrated Web Framework for Inland Waters in Iowa

“…Data pre-processing techniques, such as standardization and normalization, are used to make variables that have different scales comparable. This helps machine learning algorithms to make more accurate and consistent predictions (Fukami et al, 2021;Rubio-Herrero et al, 2022;Abdullah et al, 2022). Therefore, microcystin, dissolved oxygen and evaporation values were normalized due to their significant differences in scales.…”

Random feature models for learning interacting dynamical systems

“…polynomials or trigonometric functions) applied to the data. This approach has lead to many related algorithms and applications, see [21][22][23][24][25][26][27][28][29][30][31][32][33][34] and the reference within. The sparse optimization framework for learning governing equations was proposed in [35] along with an approach to discovering partial differential equations using a dictionary of derivatives.…”