Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

Fukushima, Kunihiko

doi:10.1007/bf00344251

Cited by 4,627 publications

(2,345 citation statements)

References 6 publications

Supporting

Mentioning

2,281

Contrasting

Unclassified

Order By: Relevance

“…5.9), although his architecture was comparatively deep indeed. For downsampling purposes he used Spatial Averaging (Fukushima, 1980(Fukushima, , 2011 instead of Max-Pooling (MP, Sec. 5.11), currently a particularly convenient and popular WTA mechanism.…”

Section: : Convolutionmentioning

confidence: 99%

“…5.11) CNNs (Fukushima, 1979;LeCun et al, 1989) Multi-Column GPU-MPCNNs (Ciresan et al, 2011b) are committees (Breiman, 1996;Schapire, 1990;Wolpert, 1992;Hashem and Schmeiser, 1992;Ueda, 2000;Dietterich, 2000a) of GPUMPCNNs with simple democratic output averaging. Several MPCNNs see the same input; their output vectors are used to assign probabilities to the various possible classes.…”

Section: : Mpcnns On Gpu Achieve Superhuman Vision Performancementioning

confidence: 99%

See 1 more Smart Citation

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

Section: : Convolutionmentioning

confidence: 99%

Section: : Mpcnns On Gpu Achieve Superhuman Vision Performancementioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

show abstract

“…Filter responses are then repeatedly pooled and refiltered, resulting in a deep feed-forward network architecture whose output feature vectors are eventually classified. One of the first hierarchical neural systems was the Neocognitron by Fukushima (1980), which inspired many of the more recent variants.…”

Section: Introductionmentioning

confidence: 99%

Multi-column deep neural network for traffic sign classification

et al. 2012

View full text Add to dashboard Cite

We describe the approach that won the final phase of the German traffic sign recognition benchmark. Our method is the only one that achieved a betterthan-human recognition rate of 99.46%. We use a fast, fully parameterizable GPU implementation of a Deep Neural Network (DNN) that does not require careful design of pre-wired feature extractors, which are rather learned in a supervised way. Combining various DNNs trained on differently preprocessed data into a Multi-Column DNN (MCDNN) further boosts recognition performance, making the system insensitive also to variations in contrast and illumination.

show abstract

“…However, for performing visual task, as well as the classification of MCCs, this approach is far from sufficient. Another type of neural network model is the Fukushima's Neocognitrons (Fukushima, 1980(Fukushima, , 1988Fukushima & Miyake, 1982;Fukushima & Wake, 1991), and evolved Tricognitron (Xu, 1993;Xu & Chang, 1996). The Fukushima's Neocognitrons consist of feature extraction, S, layers and fusion, C, layers, where the S layers are constructed either by some pre-selected shape patterns, or by an unsupervised learning to code the object features in the planes.…”

Section: Introductionmentioning

confidence: 99%

“…The Fukushima's Neocognitrons consist of feature extraction, S, layers and fusion, C, layers, where the S layers are constructed either by some pre-selected shape patterns, or by an unsupervised learning to code the object features in the planes. The unsupervised Neocognitrons were considered comprising biological neural model, and effective to the recognition of characters, invariant to size, translation and handwriting (Fukushima, 1980(Fukushima, , 1988Fukushima & Miyake, 1982;Fukushima & Wake, 1991). Due to these characters of possessing biological meaning and effective in character recognition, the Neocognitrons were regarded one type of very unique neural models.…”

Section: Introductionmentioning

confidence: 99%