Consensus-based optimization methods converge globally

Fornasier, Massimo; Klock, Timo; Riedl, Konstantin

doi:10.48550/arxiv.2103.15130

Cited by 16 publications

(65 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we believe that the analysis framework of this and prior works on CBO [36,5,14] motivates to investigate also other renowned metaheuristic algorithms through a mean-field limit.…”

Section: Discussionmentioning

confidence: 88%

“…It is also worth noting that Equation (2.1) bears a certain resemblance to CBO [36,5,6,14,15]. Indeed, as made rigorous in [7], CBO methods can be derived from PSO in the small inertia limit m → 0.…”

Section: Pso Without Memory Effectsmentioning

confidence: 99%

“…While this might have a locality flavor, the condition is generally fulfilled in practical applications. Moreover, for CBO methods there is recent work where such assumption about the initial datum is reduced to the absolute minimum [14,15].…”

Section: Convergence Of the Pso Model Without Memory Effects To A Glo...mentioning

confidence: 99%

“…To feature the latter, the two remaining terms in (1.2c), each associated with a drift term, are diffusion terms injecting randomness into the dynamics through independent standard Brownian motions (B 1,i t ) t≥0 i=1,...,N and (B 2,i t ) t≥0 i=1,...,N . The two commonly studied diffusion types for similar methods are isotropic [36,5,14] and anisotropic [6,15] diffusion with…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On the Global Convergence of Particle Swarm Optimization Methods

Huang¹,

Qiu²,

Riedl³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we provide a rigorous convergence analysis for the renowned Particle Swarm Optimization method using tools from stochastic calculus and the analysis of partial differential equations. Based on a time-continuous formulation of the particle dynamics as a system of stochastic differential equations, we establish the convergence to a global minimizer in two steps. First, we prove the consensus formation of the dynamics by analyzing the time-evolution of the variance of the particle distribution. Consecutively, we show that this consensus is close to a global minimizer by employing the asymptotic Laplace principle and a tractability condition on the energy landscape of the objective function. Our results allow for the usage of memory mechanisms, and hold for a rich class of objectives provided certain conditions of well-preparation of the hyperparameters and the initial datum are satisfied. To demonstrate the applicability of the method we propose an efficient and parallelizable implementation, which is tested in particular on a competitive and well-understood high-dimensional benchmark problem in machine learning.

show abstract

“…Finally, we believe that the analysis framework of this and prior works on CBO [36,5,14] motivates to investigate also other renowned metaheuristic algorithms through a mean-field limit.…”

Section: Discussionmentioning

confidence: 88%

Section: Pso Without Memory Effectsmentioning

confidence: 99%

Section: Convergence Of the Pso Model Without Memory Effects To A Glo...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the Global Convergence of Particle Swarm Optimization Methods

Huang¹,

Qiu²,

Riedl³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For any fixed parameter β, we need to solve an unconstrained optimization problem for which the behavior of the CBO method has been broadly analyzed, e.g. in [10,13,23,24,32,47].…”

Section: The Update Strategy For the Penalty Parametermentioning

confidence: 99%

Constrained consensus-based optimization

Borghi¹,

Herty²,

Pareschi³

2021

Preprint

View full text Add to dashboard Cite

In this work we are interested in the construction of numerical methods for high dimensional constrained nonlinear optimization problems by particle-based gradient-free techniques. A consensus-based optimization (CBO) approach combined with suitable penalization techniques is introduced for this purpose. The method relies on a reformulation of the constrained minimization problem in an unconstrained problem for a penalty function and extends to the constrained settings the class of CBO methods. Exact penalization is employed and, since the optimal penalty parameter is unknown, an iterative strategy is proposed that successively updates the parameter based on the constrained violation. Using a mean-field description of the the many particle limit of the arising CBO dynamics, we are able to show convergence of the proposed method to the minimum for general nonlinear constrained problems. Properties of the new algorithm are analyzed. Several numerical examples, also in high dimensions, illustrate the theoretical findings and the good performance of the new numerical method.

show abstract

On the mean‐field limit for the consensus‐based optimization

Huang

Qiu

2022

Math Methods in App Sciences

View full text Add to dashboard Cite

This paper is concerned with the large particle limit for the consensus‐based optimization (CBO), which was postulated in the pioneering works by Carrillo, Pinnau, Totzeck and many others. In order to solve this open problem, we adapt a compactness argument by first proving the tightness of the empirical measures false{μNfalse}N≥2$$ {\left\{{\mu}&#x0005E;N\right\}}_{N\ge 2} $$ associated to the particle system and then verifying that the time marginal of the limit measure μ$$ \mu $$ is the unique weak solution to the mean‐field CBO equation. Such results are further extended to the model of particle swarm optimization (PSO).

show abstract

Consensus-based optimization methods converge globally

Cited by 16 publications

References 27 publications

On the Global Convergence of Particle Swarm Optimization Methods

On the Global Convergence of Particle Swarm Optimization Methods

Constrained consensus-based optimization

On the mean‐field limit for the consensus‐based optimization

Contact Info

Product

Resources

About