SPGD: Search Party Gradient Descent Algorithm, a Simple Gradient-Based Parallel Algorithm for Bound-Constrained Optimization

Abstract

Nature-inspired metaheuristic algorithms remain a strong trend in optimization. Human-inspired optimization algorithms should be more intuitive and relatable. This paper proposes a novel optimization algorithm inspired by a human search party. We hypothesize the behavioral model of a search party searching for a treasure. Motivated by the search party’s behavior, we abstract the “Divide, Conquer, Assemble” (DCA) approach. The DCA approach allows us to parallelize the traditional gradient descent algorithm in a strikingly simple manner. Essentially, multiple gradient descent instances with different learning rates are run parallelly, periodically sharing information. We call it the search party gradient descent (SPGD) algorithm. Experiments performed on a diverse set of classical benchmark functions show that our algorithm is good at optimizing. We believe our algorithm’s apparent lack of complexity will equip researchers to solve problems efficiently. We compare the proposed algorithm with SciPy’s optimize library and it is found to be competent with it.

A novel reinforcement learning framework for disassembly sequence planning using Q-learning technique optimized using an enhanced simulated annealing algorithm

“…Inspired by this concept of parameter adaptation to overcome the problems of local minima and slow convergence rate, an enhanced simulated annealing (ESA) algorithm has been implemented in this work. By enhancing the convergence rates, a wide range of machine learning models can be explored to address the challenges in DSP (Syed Shahul Hameed and Rajagopalan, 2022, 2023).…”

M2F-Net: A Deep Learning-Based Multimodal Classification with High-Throughput Phenotyping for Identification of Overabundance of Fertilizers

“…To address the limited number of samples in the dataset, data augmentation techniques were utilized, which involved applying various transformations such as rotation (both horizontal and vertical), scaling, and flipping. The optimizer and loss function are the key elements that provide the network with the ability to handle large amounts of data and regulate the learning speed [49]. Different optimizers such as Adam, Stochastic Gradient Descend, and RMSprop were applied and found that the Adam optimizer has demonstrated superior performance.…”