On the premature convergence of particle swarm optimization

Larsen, Rie B.; Jouffroy, Jérôme; Lassen, Benny

doi:10.1109/ecc.2016.7810572

Cited by 12 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, once the particles move closer to the global best, which is a local optimum, they will lose an ability to search for the global optimum (Limlawan and Pongchairerks, 2010). This phenomenon is in fact a well-known problem of PSO (Larsen et al, 2016). Most of the proposed solution to mitigate this problem are problem-dependent but generally based on random-perturbation (Ünal and Kayakutlu, 2020) or hybrid approach by combining the strengths of the PSO and the other GA methods that allows for escaping local minima (Fan and Jen, 2019).…”

Section: Mitigating Underestimation Biasmentioning

confidence: 99%

On the Faithfulness of Conditional Probability Estimates: A Multi-objective Approach

Blanzeisky

Cunningham

2022

Preprint

View full text Add to dashboard Cite

Estimating conditional probabilities from data samples is the building block of many machine learning algorithms. However, these estimates often do not reflect the true underlying distribution. With ML systems becoming more ubiquitous in high-risk decision making, it is critical to investigate how the predictions made by the algorithm can misrepresent the actual data distribution. To this end, we explore the notion of underestimation bias that captures the extent to which a learned model predictions deviate from the true distribution. Since fixing underestimation bias may come at the cost of accuracy, we propose a multi-objective optimization strategy to obtain a diverse set of models with members representing different accuracy/faithfulness tradeoffs. We empirically evaluate our framework on two synthetic and twelve real-world datasets. We show that our framework can address underestimation bias while still maintaining adequate overall generalization accuracy.

show abstract

Section: Mitigating Underestimation Biasmentioning

confidence: 99%

On the Faithfulness of Conditional Probability Estimates: A Multi-objective Approach

Blanzeisky

Cunningham

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…PSO's advantages include being easy to implement, having only a few parameters to be set, having a fast convergence speed, being effective in global search, and being computationally inexpensive in terms of both memory and CPU requirements. However, even though PSO is an efficient and fast search algorithm, it also has some disadvantages, such as premature convergence problem, that is, it can be trapped into a local minimum in high-dimensional space, especially with complex problems [25].…”

Section: Proposed Algorithm Psogsmentioning

confidence: 99%

“…PSO is a fast algorithm that is easy to implement and is considered one of the most and widely used powerful algorithms for solving global optimization problems. Nevertheless, PSO has disadvantages, such as premature convergence and falling into a local minimum in high-dimensional space [25]. Similarly, GS is also easy to implement and understand, has high optimization accuracy, and has strong global optimization ability.…”

Section: Introductionmentioning

confidence: 99%

A Novel Hybrid PSO- and GS-based Hyperparameter Optimization Algorithm for Support Vector Regression

Acikkar

Altunkol

2022

Preprint

View full text Add to dashboard Cite

Hyperparameter optimization is vital in improving the prediction accuracy of Support Vector Regression (SVR), as in all machine learning algorithms. This study introduces a new hybrid optimization algorithm, namely PSOGS, which consolidates two strong and widely used algorithms, Particle Swarm Optimization (PSO) and Grid Search (GS). This hybrid algorithm was experimented on five benchmark datasets. The speed and the prediction accuracy of PSOGS-optimized SVR models (PSOGS-SVR) were compared to those of its constituent algorithms (PSO and GS) and another hybrid optimization algorithm (PSOGSA) that combines PSO and Gravitational Search Algorithm (GSA). The prediction accuracies were evaluated and compared in terms of root mean square error and mean absolute percentage error. For the sake of reliability, the results of the experiments were obtained by performing 10-fold cross-validation on 30 runs. The results showed that PSOGS-SVR yields prediction accuracy comparable to GS-SVR, performs much faster than GS-SVR, and provides better results with less execution time than PSO-SVR. Besides, PSOGS-SVR presents more effective results than PSOGSA-SVR in terms of both prediction accuracy and execution time. As a result, this study proved that PSOGS is a fast, stable, efficient, and reliable algorithm for optimizing hyperparameters of SVR.

show abstract

“…• DE and CMA-ES look, in general, to be preferable to PSO and BO; • BO is not competitive enough in both case studies and, in particular, its effectiveness deteriorates in the larger case study, probably because BO is not good enough for large dimensionalities (indeed, it is generally used to tune the few hyper-parameters of machine learning models [30]); • regarding robustness, (1 + 1)-ES is clearly the most robust algorithm; • the high variance for the results of the PSO algorithm may indicate the presence of a considerable amount of local optima in the fitness landscape, given that PSO is prone to prematurely converge to local optima [56];…”

Section: Algorithmmentioning

confidence: 99%

Using Optimisation Meta-Heuristics for the Roughness Estimation Problem in River Flow Analysis

et al. 2021

View full text Add to dashboard Cite

Climate change threats make it difficult to perform reliable and quick predictions on floods forecasting. This gives rise to the need of having advanced methods, e.g., computational intelligence tools, to improve upon the results from flooding events simulations and, in turn, design best practices for riverbed maintenance. In this context, being able to accurately estimate the roughness coefficient, also known as Manning’s n coefficient, plays an important role when computational models are employed. In this piece of research, we propose an optimal approach for the estimation of ‘n’. First, an objective function is designed for measuring the quality of ‘candidate’ Manning’s coefficients relative to specif cross-sections of a river. Second, such function is optimised to return coefficients having the highest quality as possible. Five well-known meta-heuristic algorithms are employed to achieve this goal, these being a classic Evolution Strategy, a Differential Evolution algorithm, the popular Covariance Matrix Adaptation Evolution Strategy, a classic Particle Swarm Optimisation and a Bayesian Optimisation framework. We report results on two real-world case studies based on the Italian rivers ‘Paglia’ and ‘Aniene’. A comparative analysis between the employed optimisation algorithms is performed and discussed both empirically and statistically. From the hydrodynamic point of view, the experimental results are satisfactory and produced within significantly less computational time in comparison to classic methods. This shows the suitability of the proposed approach for optimal estimation of the roughness coefficient and, in turn, for designing optimised hydrological models.

show abstract

On the premature convergence of particle swarm optimization

Cited by 12 publications

References 21 publications

On the Faithfulness of Conditional Probability Estimates: A Multi-objective Approach

On the Faithfulness of Conditional Probability Estimates: A Multi-objective Approach

A Novel Hybrid PSO- and GS-based Hyperparameter Optimization Algorithm for Support Vector Regression

Using Optimisation Meta-Heuristics for the Roughness Estimation Problem in River Flow Analysis

Contact Info

Product

Resources

About