Neocognitron trained with winner-kill-loser rule

Fukushima, Kunihiko

doi:10.1016/j.neunet.2010.04.008

Cited by 17 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hebbian-based, competitive learning has also been used in conjunction with a dictionary-building strategy in the Neocognitron model [17]. According to this scheme, existing units compete upon presentation of inputs, and the weights of the winning unit are modified according to a variant of Hebbian learning.…”

Section: Introductionmentioning

confidence: 99%

“…The WKL rule was proposed in [17] for purpose of learning higher-order combinations of features in the neocognitron model for written digit classification [24]. The WKL rule can be described as follows.…”

Section: Introductionmentioning

confidence: 99%

“…The threshold θ may be seen as defining a neighborhood in the input space, located around the center wi, for which patterns will strongly activate unit i. Reference [17] considered only the case of λ=1, although here we use λ=0.05 for all simulations. The weights for all inactive units (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…Simulations in the neocognitron have shown that the WKL compares favorably at least on a written digit recognition task [17].…”

Section: Introductionmentioning

confidence: 99%

“…Incremental learning procedures thus seem better suited to learn higher-order features formed by combinations of handcrafted oriented edge detectors like Gabor filters or Difference-of-Gaussians [4,17].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Online learning of feature detectors from natural images with the probabilistic WKL rule

Léveillé

Hayashi

Fukushima

2012

The 6th International Conference on Soft Computing and Intelligent Systems, and the 13th International Symposium on Advanced In

Self Cite

View full text Add to dashboard Cite

Recent advances in machine learning and computer vision have led to the development of several sophisticated learning schemes for object recognition by convolutional networks. One relatively simple learning rule, the Winner-KillLoser (WKL), was shown to be efficient at learning higher-order features in the neocognitron model when used in a written digit classification task. The WKL rule is one variant of incremental clustering procedures that adapt the number of cluster components to the input data. The WKL rule seeks to provide a complete, yet minimally redundant, covering of the input distribution. It is difficult to apply this approach directly to highdimensional spaces since it leads to a dramatic explosion in the number of clustering components. In this work, a small generalization of the WKL rule is proposed to learn from highdimensional data, and is shown to lead mostly to V1-like oriented cells when applied to natural images.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Simulations in the neocognitron have shown that the WKL compares favorably at least on a written digit recognition task [17].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%