Replication-based inference algorithms for hard computational problems

Alamino, Roberto C.; Neirotti, Juan Pablo; Saad, David

doi:10.1103/physreve.88.013313

Cited by 3 publications

(13 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…possible orders of presentation, but we will choose only a number n of these sequences, with n being of polynomial order in N . In previous applications [18] we observed that this is enough to considerably improve the performance of the non-replicated algorithm.…”

Section: Replicationmentioning

confidence: 78%

“…The algorithm we propose here is based on a modified version introduced in [18] of the usual MP algorithm. The introduced modifications had the objective of circumventing one particularly serious recurring problem in optimization tasks for perceptron learning in generalthe complexity of the energy landscape.…”

Section: Message Passingmentioning

confidence: 99%

“…This is done by means of an expansion which is valid in the limit of large systems. The details of this approximation were given in [18] and we therefore provide below only a brief description of its derivation.…”

Section: Message Passingmentioning

confidence: 99%

“…The method proposed in [18] can then be easily adapted to the present compression scheme. Using the non-linear activation function for the current perceptron, one can calculate analytically both the numerator and denominator of the equation for the conjugate magnetizations as a power series in K. This is accomplished by uncoupling the synaptic vectors by means of HubbardStratonovich fields, which can then be exactly integrated at the end.…”

Section: Message Passingmentioning

confidence: 99%

“…The aim of the current paper is to do exactly that; namely, to adapt a recently introduced MP algorithm [18] to bring the performance of the non-linear perceptron compression scheme closer to the theoretical limits. This algorithm, named the replicated on-line message passing algorithm (rOnMP), is based on improved inference in hard computational problems by averaging over results obtained from different solutions of replicated variable systems.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Compression by replication

2014

Self Cite

View full text Add to dashboard Cite

A recently introduced inference method based on system replication and an on-line message passing algorithm is employed to complete a previously suggested compression scheme based on a non-linear perceptron. The algorithm is shown to approach the information theoretical bounds for compression as the number of replicated systems increases, offering superior performance compared to basic message passing algorithms. In addition, the suggested method does not require fine-tuning of parameters or other complementing heuristic techniques, such as the introduction of inertia terms, to improve convergence rates to non-trivial results.

show abstract

Section: Replicationmentioning

confidence: 78%

Section: Message Passingmentioning

confidence: 99%

Section: Message Passingmentioning

confidence: 99%

Section: Message Passingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Compression by replication

2014

Self Cite

View full text Add to dashboard Cite

show abstract

Origin of the computational hardness for learning with binary synapses

Huang

Kabashima

2014

Phys. Rev. E

View full text Add to dashboard Cite

Through supervised learning in a binary perceptron one is able to classify an extensive number of random patterns by a proper assignment of binary synaptic weights. However, to find such assignments in practice is quite a nontrivial task. The relation between the weight space structure and the algorithmic hardness has not yet been fully understood. To this end, we analytically derive the Franz-Parisi potential for the binary perceptron problem by starting from an equilibrium solution of weights and exploring the weight space structure around it. Our result reveals the geometrical organization of the weight space; the weight space is composed of isolated solutions, rather than clusters of exponentially many close-by solutions. The pointlike clusters far apart from each other in the weight space explain the previously observed glassy behavior of stochastic local search heuristics.

show abstract

An interacting replica approach applied to the traveling salesman problem

Sun

Leonard

Ronhovde

et al. 2016

2016 SAI Computing Conference (SAI)

View full text Add to dashboard Cite

Abstract-We present a physics inspired heuristic method for solving combinatorial optimization problems. Our approach is specifically motivated by the desire to avoid trapping in metastable local minima-a common occurrence in hard problems with multiple extrema. Our method involves (i) coupling otherwise independent simulations of a system ("replicas") via geometrical distances as well as (ii) probabilistic inference applied to the solutions found by individual replicas. The ensemble of replicas evolves as to maximize the inter-replica correlation while simultaneously minimize the local intra-replica cost function (e.g., the total path length in the Traveling Salesman Problem within each replica). We demonstrate how our method improves the performance of rudimentary local optimization schemes long applied to the NP hard Traveling Salesman Problem. In particular, we apply our method to the well-known "k-opt" algorithm and examine two particular cases-k = 2 and k = 3. With the aid of geometrical coupling alone, we are able to determine for the optimum tour length on systems up to 280 cities (an order of magnitude larger than the largest systems typically solved by the bare k = 3 opt). The probabilistic replica-based inference approach improves k − opt even further and determines the optimal solution of a problem with 318 cities and find tours whose total length is close to that of the optimal solutions for other systems with a larger number of cities.

show abstract

Replication-based inference algorithms for hard computational problems

Cited by 3 publications

References 39 publications

Compression by replication

Compression by replication

Origin of the computational hardness for learning with binary synapses

An interacting replica approach applied to the traveling salesman problem

Contact Info

Product

Resources

About