Improving Particle Swarm Optimization Based on Neighborhood and Historical Memory for Training Multi-Layer Perceptron

Abstract

Many optimization problems can be found in scientific and engineering fields. It is a challenge for researchers to design efficient algorithms to solve these optimization problems. The Particle swarm optimization (PSO) algorithm, which is inspired by the social behavior of bird flocks, is a global stochastic method. However, a monotonic and static learning model, which is applied for all particles, limits the exploration ability of PSO. To overcome the shortcomings, we propose an improving particle swarm optimization algorithm based on neighborhood and historical memory (PSONHM). In the proposed algorithm, every particle takes into account the experience of its neighbors and its competitors when updating its position. The crossover operation is employed to enhance the diversity of the population. Furthermore, a historical memory M w is used to generate new inertia weight with a parameter adaptation mechanism. To verify the effectiveness of the proposed algorithm, experiments are conducted with CEC2014 test problems on 30 dimensions. Finally, two classification problems are employed to investigate the efficiencies of PSONHM in training Multi-Layer Perceptron (MLP). The experimental results indicate that the proposed PSONHM can effectively solve the global optimization problems.

A hybrid constriction coefficient-based particle swarm optimization and gravitational search algorithm for training multi-layer perceptron

“…Moreover, GSO has been compared with BBO, GA, and MVO algorithms for classification accuracy (Alboaneen et al, 2017). Furthermore, Li (2018) has applied an improved version of PSO on the CEC 2014 test benchmarks to find the global optimum. Besides, MLP was trained to investigate the efficiency of the proposed algorithm in handling complex search spaces.…”

“…In literature, there are many heuristic algorithms utilized for training MLPs such as differential evolution (DE) (Llonen et al, 2003;Slowik and Bialko, 2008), ant colony optimization (ACO) (Blum and Socha, 2005;Socha and Blum, 2007), genetic algorithm (GA) (Whitney et al, 1990;Mirjalili et al, 2012), artificial bee colony (ABC) (Karaboga et al, 2007;Ozturk and Karaboga, 2011) and particle swarm optimization (PSO) (Mendes et al, 2002;Gudise and Venayagamoorthy, 2003). The recent additions in the list of stochastic training algorithms include social spider optimization algorithm (SSO) (Pereira, 2014), teachinglearning based optimization (TLBO) (Uzlu et al, 2014), biogeography based optimization (BBO) (Mirjalili et al, 2014), symbiotic organisms search algorithm (SOS) (Wu et al, 2016), glowworm swarm optimization (GSO) (Alboaneen et al, 2017) and improved PSO (Li, 2018).…”

Particle Swarm Optimization for Micro-Grid Power Management and Load Scheduling

“…PSO has been successfully applied to solve a number of problems in research and application areas including problems of optimization of functions (Logenthiran et al, 2015;Abid et al, 2017), selection and classification (Too, 2019), wireless sensor networks (Cheng et al, 2018) and learning of neural networks (Li, 2018) etc.…”

“…Different algorithms are used and compared to schedule loads of a smart city (Logenthiran et al, 2012;Abid et al, 2017;Li, 2018;Celik et al, 2017). The aim is to reduce the energy bill by taking into account the consumption and production data, but also the current pricing policies and any operating constraints imposed by the micro-grid manager.…”

Artificial Neural Networks Hidden Unit and Weight Connection Optimization by Quasi-Refection-Based Learning Artificial Bee Colony Algorithm

“…In the paper, the results shows that GA outperforms BPon sonar image classification problems. Later, other metaheuristics are applied successfully for weight and bias optimization [3], [4], [5], [6]. The utilization of metaheuristic based algorithms improves the search ability of neural network training for such values of weights and biases which will reduce the classification error rate more than gradient-based approaches.…”