Characterization of the convergence of stationary Fokker–Planck learning

Berrones, Arturo

doi:10.1016/j.neucom.2008.12.042

Cited by 3 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we briefly describe the stationary Fokker-Planck sampling algorithm as introduced in [13] and further detailed in [14,11,12], where also more details about the general framework can be found. SFP sampling is based on the interrelation between the Langevin and Fokker-Planck equations that allow for the stochastic description of a given system.…”

Section: Algorithmmentioning

confidence: 99%

“…Besides that, the construction of heuristics based on the SFP algorithm in combination with a downhill simplex routine [15] was discussed. Recently [12] the asymptotic convergence properties of the SFP algorithm were studied by means of numerical experiments considering the 2 parameter Michaelewicz function and the XOR problem. Moreover, its applicability to problems that arise in the field of statistical inference was outlined by showing how the SFP algorithm can efficiently be used to perform maximum likelihood and Bayesian training of neural networks.…”

Section: Introductionmentioning

confidence: 99%

“…simulated annealing [3,4], parallel tempering Monte Carlo [5,6], extremal optimization [7,8] and genetic algorithms [9,10] provide valuable tools to locate points in the configuration space that correspond to a near-optimal or even an optimal value of the underlying cost function. Here we investigate a recently proposed heuristic for stochastic optimization, called stationary Fokker-Planck (SFP) sampling [11,12]. The basic idea is to perform a Langevin dynamics for the variables of a given system in the potential given by the cost function.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Ground states of 2D ±JIsing spin glasses obtained via stationary Fokker–Planck sampling

Melchert

Hartmann

2008

J. Stat. Mech.

View full text Add to dashboard Cite

We investigate the performance of the recently proposed stationary Fokker-Planck sampling method considering a combinatorial optimization problem from statistical physics. The algorithmic procedure relies upon the numerical solution of a linear second order differential equation that depends on a diffusion-like parameter D. We apply it to the problem of finding ground states of 2d Ising spin glasses for the ±J−Model. We consider square lattices with side length up to L = 24 with two different types of boundary conditions and compare the results to those obtained by exact methods. A particular value of D is found that yields an optimal performance of the algorithm. We compare this optimal value of D to a percolation transition, which occurs when studying the connected clusters of spins flipped by the algorithm. Nevertheless, even for moderate lattice sizes, the algorithm has more and more problems to find the exact ground states. This means that the approach, at least in its standard form, seems to be inferior to other approaches like parallel tempering.

show abstract

Section: Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Ground states of 2D ±JIsing spin glasses obtained via stationary Fokker–Planck sampling

Melchert

Hartmann

2008

J. Stat. Mech.

View full text Add to dashboard Cite

show abstract

“…Under general conditions, the Gibbs sampler converges at a geometric rate (Roberts & Polson, 1994;Canty, 1999) and there is some evidence that this fast convergence is shared by SFP (Berrones, 2009). In the next section the properties of the SFP sampler are studied in the wider context of Monte Carlo methods.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Inference Based on Stationary Fokker-Planck Sampling

Berrones

2010

Neural Computation

Self Cite

View full text Add to dashboard Cite

A novel formalism for bayesian learning in the context of complex inference models is proposed. The method is based on the use of the stationary Fokker-Planck (SFP) approach to sample from the posterior density. Stationary Fokker-Planck sampling generalizes the Gibbs sampler algorithm for arbitrary and unknown conditional densities. By the SFP procedure, approximate analytical expressions for the conditionals and marginals of the posterior can be constructed. At each stage of SFP, the approximate conditionals are used to define a Gibbs sampling process, which is convergent to the full joint posterior. By the analytical marginals efficient learning methods in the context of artificial neural networks are outlined. Offline and incremental bayesian inference and maximum likelihood estimation from the posterior are performed in classification and regression examples. A comparison of SFP with other Monte Carlo strategies in the general problem of sampling from arbitrary densities is also presented. It is shown that SFP is able to jump large low-probability regions without the need of a careful tuning of any step-size parameter. In fact, the SFP method requires only a small set of meaningful parameters that can be selected following clear, problem-independent guidelines. The computation cost of SFP, measured in terms of loss function evaluations, grows linearly with the given model's dimension.

show abstract

Stationary Fuzzy Fokker–Planck Learning and Stochastic Fuzzy Filtering

Kumar¹,

Stoll

2011

IEEE Trans. Fuzzy Syst.

View full text Add to dashboard Cite

Characterization of the convergence of stationary Fokker–Planck learning

Cited by 3 publications

References 33 publications

Ground states of 2D ±JIsing spin glasses obtained via stationary Fokker–Planck sampling

Ground states of 2D ±JIsing spin glasses obtained via stationary Fokker–Planck sampling

Bayesian Inference Based on Stationary Fokker-Planck Sampling

Stationary Fuzzy Fokker–Planck Learning and Stochastic Fuzzy Filtering

Contact Info

Product

Resources

About