Optimal colored perceptrons

Bollé, Désiré; Kozłowski, P.

doi:10.1103/physreve.64.011915

Cited by 4 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is extremely difficult to find points touching the axis in the simulations because of the final size effects. Analogous results have been found for the Hopfield model, the Q-Ising model [15], [16], [17] and, first, in the infinite range spin-glass [18]. Fig.…”

Section: Numerical Results and Simulationssupporting

confidence: 82%

Parallel dynamics of the fully connected Blume–Emery–Griffiths neural network

Bollé

Blanco

Shim

2003

Physica A: Statistical Mechanics and its Applications

Self Cite

View full text Add to dashboard Cite

The parallel dynamics of the fully connected Blume-Emery-Griffiths neural network model is studied at zero temperature using a probabilistic approach. A recursive scheme is found determining the complete time evolution of the order parameters, taking into account all feedback correlations. It is based upon the evolution of the distribution of the local field, the structure of which is determined in detail. As an illustrative example explicit analytic formula are given for the first few time steps of the dynamics. Furthermore, equilibrium fixed-point equations are derived and compared with the thermodynamic approach. The analytic results find excellent confirmation in extensive numerical simulations.Recently, an optimal Hamiltonian has been derived in the statistical mechanics approach to Qstate neural networks starting from the concept of mutual information [1], [2]. Optimal means that the best retrieval properties are guaranteed including, e.g., the largest retrieval overlap, loading capacity, basin of attraction, convergence time. For Q = 3 this Hamiltonian resembles the classical Blume-Emery-Griffiths (BEG) Hamiltonian in the sense that it contains both a bilinear and biquadratic term in the spins [3]. For a fully connected architecture it has been shown, using a thermodynamic replica approach that the maximal loading capacity for the BEG network is indeed bigger than the one for other three-state networks existing in the literature (Ising, Potts...) [2].

show abstract

Section: Numerical Results and Simulationssupporting

confidence: 82%

Parallel dynamics of the fully connected Blume–Emery–Griffiths neural network

Bollé

Blanco

Shim

2003

Physica A: Statistical Mechanics and its Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…These results clarify the resemblance found before [4,5,12] of the thermodynamic properties of the AT and the Potts neural networks. Furthermore, they imply that the four-state Potts model described by H V Z is thermodynamically equivalent to the AT neural network model with one condensed pattern only in the limit of low loading, i.e.…”

supporting

confidence: 75%

Equivalence of the Ashkin-Teller and the four-state Potts-glass models of neural networks

Bollé

Kozłowski

2001

Phys. Rev. E

Self Cite

View full text Add to dashboard Cite

We show that for a particular choice of the coupling parameters the Ashkin-Teller spin-glass neural network model with the Hebb learning rule and one condensed pattern yields the same thermodynamic properties as the four-state anisotropic Potts-glass neural network model. This equivalence is not seen at the level of the Hamiltonians. 05.50+q, 64.60.Cn, 84.35 It is well-known that the classical Ashkin-Teller (AT) model is a generalization of the Ising, the four-state clock and the four-state Potts models. This can be easily seen already at the level of the Hamiltonian, especially when one rewrites the Hamiltonian of the AT model [1] using two Ising spins located at each site of the lattice interacting via two-and four-spin couplings [2].For spin-glass systems similar observations can be made ( [3] and references therein). The AT spin-glass Hamiltonian contains as particular limits, for certain bond realizations, both the four-state clock spin glass and the four-state Potts-glass Hamiltonians.Concerning neural network models, the situation is more complicated. It is straightforward to see at the level of the Hamiltonian that for two, respectively one of the coupling strengths taken to be zero, the AT neural network model [4,5] is equivalent to the Hopfield model [6] respectively the four-state clock neural network model [7]. On the contrary, the possible relation with the fourstate Potts neural network models existing in the literature [8,9] is, at first sight, unclear. However, since we discovered in the study of the thermodynamic and retrieval properties of the AT neural network [4,5] for equal coupling strengths some resemblance to the properties of the Potts-glass neural network [8,10], we expect that a relation with the latter does exist. To investigate this relation is the purpose of this brief report.The AT neural network with the Hebb learning rule is described by the following infinite-range Hamiltonian

show abstract

“…The mapping (1)-( 2) can be equivalently represented by the set of three equations (cfr. model I in [12])…”

Section: The Model and The Learning Scenariosmentioning

confidence: 99%

“…In this paper we study on-line learning and generalisation in a recently introduced model, allowing two different types of binary neurons at each site, possibly having different functions [11,12]. More specifically, this so called Ashkin-Teller (AT) perceptron contains, besides two-neuron interaction terms, also a four-neuron interaction term.…”

Section: Introductionmentioning

confidence: 99%

“…For the underlying biological motivation for the introduction of different types of neurons we refer to [13]. Here, we recall that the maximal capacity of the AT perceptron model II introduced in [11,12] can be larger than the one of the standard binary perceptron [12] and that the corresponding recurrent network model can be a more efficient associative memory than a sum of two Hopfield models [13]. A natural question is then how this AT perceptron performs in on-line learning and generalisation tasks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On-line learning and generalization in coupled perceptrons

Bollé¹,

Kozłowski²

2002

J. Phys. A: Math. Gen.

View full text Add to dashboard Cite

We study supervised learning and generalisation in coupled perceptrons trained on-line using two learning scenarios. In the first scenario the teacher and the student are independent networks and both are represented by an Ashkin-Teller perceptron. In the second scenario the student and the teacher are simple perceptrons but are coupled by an Ashkin-Teller type four-neuron interaction term. Expressions for the generalisation error and the learning curves are derived for various learning algorithms. The analytic results find excellent confirmation in numerical simulations.

show abstract

Optimal colored perceptrons

Abstract: Ashkin-Teller type perceptron models are introduced. Their maximal capacity per number of couplings is calculated within a first-step replica-symmetry-breaking Gardner approach. The results are compared with extensive numerical simulations using several algorithms.

Cited by 4 publications

References 20 publications

Parallel dynamics of the fully connected Blume–Emery–Griffiths neural network

Parallel dynamics of the fully connected Blume–Emery–Griffiths neural network

Equivalence of the Ashkin-Teller and the four-state Potts-glass models of neural networks

On-line learning and generalization in coupled perceptrons

Contact Info

Product

Resources

About