Convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization

Regis, Rommel G.

doi:10.1016/j.ejor.2010.07.005

Cited by 14 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, a novel version of the q -G method, which is able to guarantee the convergence of the algorithm to the global minimum in a probabilistic sense, is under development. This version is based on the generalized adaptive random search (GARS) framework for deterministic functions (Regis 2010 ). In addition, gains in the performance of the q -G method are expected with the implementation of several improvements, such as inclusion of side, linear and nonlinear restrictions, development of better step selection strategies and others.…”

Section: Discussionmentioning

confidence: 99%

The q-G method

et al. 2015

View full text Add to dashboard Cite

In this work, the q-Gradient (q-G) method, a q-version of the Steepest Descent method, is presented. The main idea behind the q-G method is the use of the negative of the q-gradient vector of the objective function as the search direction. The q-gradient vector, or simply the q-gradient, is a generalization of the classical gradient vector based on the concept of Jackson’s derivative from the q-calculus. Its use provides the algorithm an effective mechanism for escaping from local minima. The q-G method reduces to the Steepest Descent method when the parameter q tends to 1. The algorithm has three free parameters and it is implemented so that the search process gradually shifts from global exploration in the beginning to local exploitation in the end. We evaluated the q-G method on 34 test functions, and compared its performance with 34 optimization algorithms, including derivative-free algorithms and the Steepest Descent method. Our results show that the q-G method is competitive and has a great potential for solving multimodal optimization problems.

show abstract

Section: Discussionmentioning

confidence: 99%

The q-G method

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Inspired by [21], the next theorem deals with the case where f has a unique global minimiser x * over D. In this situation, the sequence of best solutions {X k } k∈N converges to x * almost surely. Theorem 6.2.…”

Section: The Convergence Of the P-lbfgsb Algorithmmentioning

confidence: 99%

A Perturbed Quasi-Newton Algorithm for Bound-Constrained Global Optimization

null,

Bencherif-Madani

2025

JCM

View full text Add to dashboard Cite

This paper presents a stochastic modification of a limited memory BFGS method to solve bound-constrained global minimization problems with a differentiable cost function with no further smoothness. The approach is a stochastic descent method where the deterministic sequence, generated by a limited memory BFGS method, is replaced by a sequence of random variables. To enhance the performance of the proposed algorithm and make sure the perturbations lie within the feasible domain, we have developed a novel perturbation technique based on truncating a multivariate double exponential distribution to deal with bound-constrained problems; the theoretical study and the simulation of the developed truncated distribution are also presented. Theoretical results ensure that the proposed method converges almost surely to the global minimum. The performance of the algorithm is demonstrated through numerical experiments on some typical test functions as well as on some further engineering problems. The numerical comparisons with stochastic and meta-heuristic methods indicate that the suggested algorithm is promising.

show abstract

“…2.1]. Due to stochastic nature of the algorithm, the iterates are treated as random vectors whose realizations are in D, following the ideas of [35]. The algorithm is general since the iterates are given randomly by any probability distribution.…”

Section: General Filter-based Stochastic Algorithmmentioning

confidence: 99%

“…Now we analyse the convergence of Algorithm 1 in the probabilistic sense following the ideas of [35]. We say that an algorithm converges to the global minimum of f on D in probability, or almost surely, if the sequence (f (X * k )) converges to f * in probability, or almost surely.…”

Section: 1]mentioning

confidence: 99%

See 1 more Smart Citation

Filter-based stochastic algorithm for global optimization

et al. 2020

View full text Add to dashboard Cite

We propose the general Filter-based Stochastic Algorithm (FbSA) for the global optimization of nonconvex and nonsmooth constrained problems. Under certain conditions on the probability distributions that generate the sample points, almost sure convergence is proved. In order to optimize problems with computationally expensive black-box objective functions, we develop the FbSA-RBF algorithm based on the general FbSA and assisted by Radial Basis Function (RBF) surrogate models to approximate the objective function. At each iteration, the resulting algorithm constructs/updates a surrogate model of the objective function and generates trial points using a dynamic coordinate search strategy similar to the one used in the Dynamically Dimensioned Search method. To identify a promising best trial point, a non-dominance concept based on the values of the surrogate model and the constraint violation at the trial points is used. Theoretical results concerning the sufficient conditions for the almost surely convergence of the algorithm are presented. Preliminary numerical experiments show that the FbSA-RBF is competitive when compared with other known methods in the literature.

show abstract

Convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization

Cited by 14 publications

References 13 publications

The q-G method

The q-G method

A Perturbed Quasi-Newton Algorithm for Bound-Constrained Global Optimization

Filter-based stochastic algorithm for global optimization

Contact Info

Product

Resources

About