Storage capacity of a constructive learning algorithm

Buhot, Arnaud; Gordon, Mirta B.

doi:10.1088/0305-4470/33/9/301

Cited by 5 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For different possible choices of the potential V in (2), we deduce the number of hidden units, k(α; V ) and the algorithm storage capacity of the TLA. Only the main results are presented here, the interested reader can find the details in [11,12].…”

Section: Results For Different Learning Potentialsmentioning

confidence: 99%

“…where E t (α, ε i−1 t ) is the training error of a simple perceptron trained with a training set of size α and biased targets τ i µ drawn with a probability P (τ i µ ; ε i−1 t ) given by (1). The number k of perceptrons necessary to correctly classify the initial training set satisfies [11,12]:…”

Section: Storage Capacitymentioning

confidence: 99%

“…It is thus possible to use the continuum limit: with x = i/k. After integration of this differential equation we obtain [11,12] the typical number of hidden units introduced by the TLA:…”

Section: Storage Capacitymentioning

confidence: 99%

See 2 more Smart Citations

Storage capacity of the Tilinglike Learning Algorithm

Buhot¹,

Gordon²

2001

AIP Conference Proceedings

Self Cite

View full text Add to dashboard Cite

The storage capacity of an incremental learning algorithm for the parity machine, the Tilinglike Learning Algorithm, is analytically determined in the limit of a large number of hidden perceptrons. Different learning rules for the simple perceptron are investigated. The usual Gardner-Derrida one leads to a storage capacity close to the upper bound, which is independent of the learning algorithm considered.

show abstract

Section: Results For Different Learning Potentialsmentioning

confidence: 99%

Section: Storage Capacitymentioning

confidence: 99%

See 1 more Smart Citation

Storage capacity of the Tilinglike Learning Algorithm

Buhot¹,

Gordon²

2001

AIP Conference Proceedings

Self Cite

View full text Add to dashboard Cite

show abstract

“…& Honavar, 1995) (Biehl & Opper, 1991) presents a tiling-like constructive algorithm for a parity-machine. (Buhot & Gordon, 2000) derives upper and lower bounds for the typical storage capacity of this constructive algorithm.…”

Section: Discussionmentioning

confidence: 99%

The constraint based decomposition (CBD) training architecture

Drăghici

2001

Neural Networks

View full text Add to dashboard Cite

The Constraint Based Decomposition (CBD) is a constructive neural network technique that builds a three or four layer neural network and is guaranteed to find a solution for any classification problem. CBD is shown to be able to solve complicated problems in a simple, fast and reliable manner. The technique is further enhanced by two modifications (locking detection and redundancy elimination) which address the training speed and the efficiency of the internal representation built by the network. The redundancy elimination aims at building more compact architectures while the locking detection aims at improving the training speed. The computational cost of the redundancy elimination is negligible and this enhancement can be used for any problem. However, the computational cost of the locking detection is exponential in the number of dimensions and should only be used in low dimensional spaces. The experimental results show the performance of the algorithm presented in a series of classical benchmark problems including the 2-spiral problem and the Iris, Wine, Glass, Lenses, Ionosphere, Lung cancer, Pima indians, Bupa, Tic-Tac-Toe, Balance and Zoo data sets from the CMU machine learning repository. CBD's generalization accuracy is compared with that of C4.5, C4.5 with rules, incremental decision trees, oblique classifiers, linear machine decision trees, CN2, learning vector quantization (LVQ), backpropagation, nearest neighbor, Q* and radial basis functions (RBFs). CBD provides the second best average accuracy on the problems tested as well as the best reliability (the lowest standard deviation).

show abstract

“…In the case of SVM's, the former is the feature space. Since the number M of training patterns remains fixed, one can wonder whether SVM's are able to generalize at all [Buhot et al 2000]. It has been proved that the generalization error of the SVM's is bounded by the fraction of training patterns that are support vectors.…”

Section: Remarkmentioning

confidence: 99%

Discrimination

Gordon¹

Neural Networks

View full text Add to dashboard Cite

Storage capacity of a constructive learning algorithm

Cited by 5 publications

References 22 publications

Storage capacity of the Tilinglike Learning Algorithm

Storage capacity of the Tilinglike Learning Algorithm

The constraint based decomposition (CBD) training architecture

Discrimination

Contact Info

Product

Resources

About