On-line learning in the Ising perceptron

Rosen-Zvi, Michal

doi:10.1088/0305-4470/33/41/302

Cited by 6 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This assumption is violated in case that the updating of the continuous vector itself is made according to the clipped one, see [6,11]. The results are:…”

Section: The Order Parametersmentioning

confidence: 99%

Training a perceptron in a discrete weight space

Rosen-Zvi

Kanter

2001

Phys. Rev. E

Self Cite

View full text Add to dashboard Cite

On-line and batch learning of a perceptron in a discrete weight space, where each weight can take 2L + 1 different values, are examined analytically and numerically. The learning algorithm is based on the training of the continuous perceptron and prediction following the clipped weights. The learning is described by a new set of order parameters, composed of the overlaps between the teacher and the continuous/clipped students. Different scenarios are examined among them on-line learning with discrete/continuous transfer functions and off-line Hebb learning. The generalization error of the clipped weights decays asymptotically as exp(−Kα 2 )/exp(−e |λ|α ) in the case of on-line learning with binary/continuous activation functions, respectively, where α is the number of examples divided by N, the size of the input vector and K is a positive constant that decays linearly with 1/L. For finite N and L, a perfect agreement between the discrete student and the teacher is obtained for α ∝ L ln(N L). A crossover to the generalization error ∝ 1/α, characterized continuous weights with binary output, is obtained for synaptic depth L > O( √ N ).

show abstract

“…This assumption is violated in case that the updating of the continuous vector itself is made according to the clipped one, see [6,11]. The results are:…”

Section: The Order Parametersmentioning

confidence: 99%

Training a perceptron in a discrete weight space

Rosen-Zvi

Kanter

2001

Phys. Rev. E

Self Cite

View full text Add to dashboard Cite

show abstract

“…Algorithms and mathematical models for perceptron-based supervised learning can encompass a ‘teacher’ element that provides data sets and determines responses to those data, and a ‘student’ element, whose learning is directed by the teacher [15]. The biological student–teacher (BST) network consists of sets of genes within teacher and student cells that interact via promoting or repressing outputs.…”

Section: Introductionmentioning

confidence: 99%

Synthetic biology routes to bio-artificial intelligence

Nesbeth

Zaikin

Saka

et al. 2016

Essays in Biochemistry

View full text Add to dashboard Cite

The design of synthetic gene networks (SGNs) has advanced to the extent that novel genetic circuits are now being tested for their ability to recapitulate archetypal learning behaviours first defined in the fields of machine and animal learning. Here, we discuss the biological implementation of a perceptron algorithm for linear classification of input data. An expansion of this biological design that encompasses cellular ‘teachers’ and ‘students’ is also examined. We also discuss implementation of Pavlovian associative learning using SGNs and present an example of such a scheme and in silico simulation of its performance. In addition to designed SGNs, we also consider the option to establish conditions in which a population of SGNs can evolve diversity in order to better contend with complex input data. Finally, we compare recent ethical concerns in the field of artificial intelligence (AI) and the future challenges raised by bio-artificial intelligence (BI).

show abstract

“…The SBPI learning rules and hidden states requirements are the same as those of the well known clipped perceptron algorithm (CP, see e.g. [17]), the only difference being an additional, purely metaplastic rule, which is only applied if the answer given by the device is correct, but such that a single variable flip would result in a classification error.…”

Section: Introductionmentioning

confidence: 99%

“…This problem is in fact easier to address than the classification problem, and optimal algorithms can be found which solve it in the binary synapses case as well (see e.g. [8,17,19]); however, such algorithms are not suitable to be considered as candidates for biological models for online learning, being either too complex or requiring to perform all intermediate operations with an auxiliary device with continuous synaptic weights. The resulting differential equation set that we obtained gives some insight on the learning dynamics and about the reason for SBPI's effectiveness, and allows for a further simplification of the SBPI algorithm, yielding an even more attractive model of neuronal unit, both from the point of view of biological feasibility and of hardware manufacturing design simplicity.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generalization Learning in a Perceptron with Binary Synapses

Baldassi

2009

J Stat Phys

View full text Add to dashboard Cite

We consider the generalization problem for a perceptron with binary synapses, implementing the Stochastic BeliefPropagation-Inspired (SBPI) learning algorithm which we proposed earlier, and perform a mean-field calculation to obtain a differential equation which describes the behaviour of the device in the limit of a large number of synapses N . We show that the solving time of SBPI is of order N √ log N , while the similar, well-known clipped perceptron (CP) algorithm does not converge to a solution at all in the time frame we considered. The analysis gives some insight into the ongoing process and shows that, in this context, the SBPI algorithm is equivalent to a new, simpler algorithm, which only differs from the CP algorithm by the addition of a stochastic, unsupervised meta-plastic reinforcement process, whose rate of application must be less than 2/ (πN ) for the learning to be achieved effectively. The analytical results are confirmed by simulations.

show abstract

On-line learning in the Ising perceptron

Cited by 6 publications

References 20 publications

Training a perceptron in a discrete weight space

Training a perceptron in a discrete weight space

Synthetic biology routes to bio-artificial intelligence

Generalization Learning in a Perceptron with Binary Synapses

Contact Info

Product

Resources

About