A comparison of general-purpose optimization algorithms for finding optimal approximate experimental designs

Abstract

Several common general purpose optimization algorithms are compared for finding A-and D-optimal designs for different types of statistical models of varying complexity, including high dimensional models with five and more factors. The algorithms of interest include exact methods, such as the interior point method, the Nelder-Mead method, the active set method, the sequential quadratic programming, and metaheuristic algorithms, such as particle swarm optimization, simulated annealing and genetic algorithms. Several simulations are performed, which provide general recommendations on the utility and performance of each method, including hybridized versions of metaheuristic algorithms for finding optimal experimental designs. A key result is that general-purpose optimization algorithms, both exact methods and metaheuristic algorithms, perform well for finding optimal approximate experimental designs.

KPLS Optimization With Nature-Inspired Metaheuristic Algorithms

“…These results are consistent with those of previous publications [8], which suggest that deterministic algorithms are more computationally efficient than nature-inspired metaheuristic algorithms and refer to the slow convergence of metaheuristic algorithms.…”

“…The distributions of the distribution of σ parameter estimates by the natureinspired metaheuristics algorithm are presented in Figure 2 for both datasets. According to [8], in a complex optimization problem where there is no theory to find or verify the optimal, if several nature-inspired metaheuristic algorithms with tuned hyperparameters converge to a solution, this solution is most likely the optimal solution.…”

“…In recent decades, so-called nature-inspired metaheuristic algorithms have emerged, which are powerful and effective in addressing various types of optimization problems, do not require assumptions to work well and are flexible and easy to implement [8]. In addition, according to Lin et al [9], metaheuristic algorithms obtain better search subsets to determine optimal or approximate solutions in the transformed space, so they are suitable for determining both the kernel function parameters and number of components in the KPLS regression [10].…”

Review on Nature-Inspired Algorithms

“…In [30], the results showed that SA produced better results than tailored heuristic algorithm in solving the job shop scheduling problem. SA has been used in optimal design problems where many researchers consider SA as a tool in the development process of optimal experimental design [35][36][37][38].…”

Multidisciplinary collaborative optimization based on relaxation method for solving complex problems

“…During the last three decades, several search algorithms have been proposed and implemented to solve optimization problems in many engineering and industrial applications; Some are exact with mathematical proofs of convergence and others are based on metaheuristics (Garcı´a-Ro´denas et al, 2020). The exact optimization algorithms include sequential quadratic programming (SQP), interior-point methods (IPMs), and the active set method (ASM).…”