Neocognitron: A neural network model for a mechanism of visual pattern recognition

Abstract

A neural network model, called a "neocognitron," for a mechanism of visual pattern recognition was proposed earlier, and the result of computer simulation for a small-scale network was shown. A neocognitron with a larger-scale network is now simulated on a digital computer and is shown to have a great capability for visual pattern recognition: The neocognitron's ability to recognize handwritten Arabic numerals, even with considerable deformations in shape, is demonstrated. The neocognitron is a multilayered network consisting of a cascaded connection of many layers of cells. The information of the stimulus pattern given to the input layer is processed step by step in each stage of the multilayered network. A cell in a deeper layer generally has a tendency to respond selectively to a more complicated feature of the stimulus patterns and, at the same time, has a larger receptive field and is less sensitive to shifts in position of the stimulus patterns. Thus each cell of the deepest layer of the network responds selectively to a specific stimulus pattern and is not affected by the distortion in shape or the shift in position of the pattern. The synapses between the cells in the network are modifiable, and the neocognitron has a function of learning. A learning-with-a-teacher process is used to reinforce these modifiable synapses in the new model, instead of the learning-without-a-teacher process which was applied to the previous small-scale model.

Implementation of Massive Artificial Neural Networks with CUDA

“…The first experiment tried to implement a neocognitron. This is an ANN designed to recognize written text (Fukushima et al, 1983). The description of neocognitron presented in the (Fausett, 1994) was used.…”

Using surfaces and surface relations in an Early Cognitive Vision system

“…In a number of works in computer vision, the potential of hierarchical structures were successfully exploited [12,17,18,20,27,30,61,[70][71][72]84]. Such approaches can be distinguished between designed [20,30] and learned [17,18,71,72] and hybrid models [12].…”

Information Routing, Correspondence Finding, and Object Recognition in the Brain

“…This idea was first proposed by Frank Rosenblatt in his four-layer perceptron (Rosenblatt 1961), and a multitude of similar models has followed since (Fukushima et al 1983, LeCun et al 1989, Mel 1997, Riesenhuber and Poggio 1999, Deco and Rolls 2004. This convergence (also called pooling) of features at different positions, scales, etc., makes the response of the following complex unit invariant to those transformations.…”

“…Feature-based approaches are very successful in classification tasks. The "standard model" from the Poggio lab Poggio 1999, Serre et al 2007) is a refinement of the Neocognitron (Fukushima et al 1983). It uses two successive stages of pooling and feature extraction and a support vector machine as a final classifier.…”

“…Pure bag-of-features models are related to the feature-based approach in its crudest form. Pyramid approaches with bags of features (Lazebnik et al 2006) introduce a bit of spatial ordering to the features and correspond to multilayer feature hierarchies like (Fukushima et al 1983) and subsequent models.…”