Parallel and interacting stochastic approximation annealing algorithms for global optimisation

Abstract

We present the parallel and interacting stochastic approximation annealing (PISAA) algorithm, a stochastic simulation procedure for global optimisation, that extends and improves the stochastic approximation annealing (SAA) by using population Monte Carlo ideas. The standard SAA algorithm guarantees convergence to the global minimum when a square-root cooling schedule is used; however the efficiency of its performance depends crucially on its self-adjusting mechanism. Because its mechanism is based on information obtained from only a single chain, SAA may present slow convergence in complex optimisation problems. The proposed algorithm involves simulating a population of SAA chains that interact each other in a manner that ensures significant improvement of the self-adjusting mechanism and better exploration of the sampling space. Central to the proposed algorithm are the ideas of (i) recycling information from the whole population of Markov chains to design a more accurate/stable self-adjusting mechanism and (ii) incorporating more advanced proposals, such as crossover operations, for the exploration of the sampling space. PISAA presents a significantly improved performance in terms of convergence. PISAA can be implemented in parallel computing environments if available. We demonstrate the good performance of the proposed algorithm on challenging applications including Bayesian network learning and protein folding. Our numerical comparisons suggest that PISAA outperforms the simulated annealing, stochastic approximation annealing, and annealing evolutionary stochastic approximation Monte Carlo especially in high dimensional or rugged scenarios.

Calibrations and validations of biological models with an application on the renal fibrosis

“…The BGO algorithm ran for 51 iterations. The EI was optimized by PISAA optimizer. In particular, EI was evaluated at 100 random locations and the “best” values were used as seeds for the PISAA optimizer which performed 1000 iterations targeting IE.…”

“…This is essentially an Empirical Bayes approach which can be seen as a fast approximation of the fully Bayesian method. To solve (6), one can implement any optimization algorithm, here we use the Parallel and Interacting Stochastic Approximation Annealing (PISAA) algorithm …”

“…It is worth mentioning that although evaluating and optimizing f (·) directly can be computationally expensive, computing and optimizing EI (9) is cheap. EI can be optimized with an existing optimization algorithm, here we use the PISAA …”

Multiple-Try Simulated Annealing Algorithm for Global Optimization

“…for = 1 to − 1 do (13) samplex j from (x * , ⋅); (14) = + (x j ); (15) = + (x); (16) set = min{ / , 1}; (17) sample u from uniform distribution in [0, 1]; (18) if < then (19) set…”

“…Such algorithm can work with a square-root cooling schedule in which the temperature can decrease much faster than that in a "logarithmic" cooling schedule. Karagiannis et al (2017) extended the SAA algorithm by using population Monte Carlo ideas and introduced the parallel and interacting stochastic approximation annealing (PISAA) algorithm [15].…”