A Hybrid Competitive Evolutionary Neural Network Optimization Algorithm for a Regression Problem in Chemical Engineering

Abstract

Neural networks have demonstrated their usefulness for solving complex regression problems in circumstances where alternative methods do not provide satisfactory results. Finding a good neural network model is a time-consuming task that involves searching through a complex multidimensional hyperparameter and weight space in order to find the values that provide optimal convergence. We propose a novel neural network optimizer that leverages the advantages of both an improved evolutionary competitive algorithm and gradient-based backpropagation. The method consists of a modified, hybrid variant of the Imperialist Competitive Algorithm (ICA). We analyze multiple strategies for initialization, assimilation, revolution, and competition, in order to find the combination of ICA steps that provides optimal convergence and enhance the algorithm by incorporating a backpropagation step in the ICA loop, which, together with a self-adaptive hyperparameter adjustment strategy, significantly improves on the original algorithm. The resulting hybrid method is used to optimize a neural network to solve a complex problem in the field of chemical engineering: the synthesis and swelling behavior of the semi- and interpenetrated multicomponent crosslinked structures of hydrogels, with the goal of predicting the yield in a crosslinked polymer and the swelling degree based on several reaction-related input parameters. We show that our approach has better performance than other biologically inspired optimization algorithms and generates regression models capable of making predictions that are better correlated with the desired outputs.

An Enhanced Fuzzy Hybrid of Fireworks and Grey Wolf Metaheuristic Algorithms

“…This distance is needed to update the position of each wolf in the next iteration using Equation (9).…”

“…Soft hybridization is the combination of two or more similar computing systems. On the contrary, the combination of two or more different computing systems can be called strong hybridization [9].…”

Modeling of Fuzzy Systems Based on the Competitive Neural Network

“…In [22], a Competitive Neural Network is used to estimate a rice-plant area; the authors demonstrate that these kinds of models are useful for the classification of the satellite data. In hybrid methods, in [23], the CNN is optimized and is applied to solve a complex problem in the field of chemical engineering; the authors proposed a novel neural network optimizer that leverages the advantages of both an improved evolutionary competitive algorithm and gradient-based backpropagation. In [24], the Fireworks Algorithm (FWA) was implemented to optimize the neurons of the CNN and improve the results of the traditional model.…”

Development of an Algorithm for Multicriteria Optimization of Deep Learning Neural Networks

“…Often, the task of neural network self-configuration is a multicriteria optimization problem [22], for which solving evolutionary algorithms, a class of stochastic algorithms that simulate the process of natural evolution, is applied [23]. Genetic algorithms (GAs) are among the most demanded evolutionary search techniques [24]. Due to their intrinsic parallelism, these algorithms make it possible to find a set of Pareto-optimal solutions in a single algorithm run [25].…”

“…The genetic algorithm provided more stable system results than the lattice search. Gavrilescu et al [24] used evolutionary algorithms to configure the properties of neural networks in their study. Akhmedova & Semenkin [25] developed a unique collective algorithm combining various bionic techniques.…”