A neural network model, called a "neocognitron", is proposed for a mechanism of visual pattern recognition. It is demonstrated by computer simulation that the neocognitron has characteristics similar to those of visual systems of vertebrates.The neocognitron is a multilayered network consisting of a cascade connection of many layers of cells, and the efficiencies of the synaptic connections between cells are modifiable. Self-organization of the network progresses by means of "learning-without-a-teacher" process: Only repetitive presentation of a set of stimulus patterns is necessary for the self-organization of the network, and no information about the categories to which these patterns should be classified is needed. The neocogni tron by itself acquires the abili tyto classify and correctly recognize these patterns according to the differences in their shapes: Any patterns which we human beings judge to be alike are also judged to be of the same category by the neocognitron. The neocognitron recognizes stimulus patterns correctly without being affected by shifts

No abstract

A neural network model, called a "neocognitron," for a mechanism of visual pattern recognition was proposed earlier, and the result of computer simulation for a small-scale network was shown. A neocognitron with a larger-scale network is now simulated on a digital computer and is shown to have a great capability for visual pattern recognition: The neocognitron's ability to recognize handwritten Arabic numerals, even with considerable deformations in shape, is demonstrated. The neocognitron is a multilayered network consisting of a cascaded connection of many layers of cells. The information of the stimulus pattern given to the input layer is processed step by step in each stage of the multilayered network. A cell in a deeper layer generally has a tendency to respond selectively to a more complicated feature of the stimulus patterns and, at the same time, has a larger receptive field and is less sensitive to shifts in position of the stimulus patterns. Thus each cell of the deepest layer of the network responds selectively to a specific stimulus pattern and is not affected by the distortion in shape or the shift in position of the pattern. The synapses between the cells in the network are modifiable, and the neocognitron has a function of learning. A learning-with-a-teacher process is used to reinforce these modifiable synapses in the new model, instead of the learning-without-a-teacher process which was applied to the previous small-scale model.

We propose a new multilayered neural network model which has the ability of rapid self-organization. This model is a modified version of the cognitron (Fukushima, 1975). It has modifiable inhibitory feedback connections, as well as conventional modifiable excitatory feedforward connections, between the cells of adjoining layers. If a feature-extracting cell in the network is excited by a stimulus which is already familiar to the network, the cell immediately feeds back inhibitory signals to its presynaptic cells in the preceding layer, which suppresses their response. On the other hand, the feature-extracting cell does not respond to an unfamiliar feature, and the responses from its presynaptic cells are therefore not suppressed because they do not receive any feedback inhibition. Modifiable synapses in the new network are reinforced in a way similar to those in the cognitron, and synaptic connections from cells yielding a large sustained output are reinforced. Since familiar stimulus features do not elicit a sustained response from the cells of the network, only circuits which detect novel stimulus features develop. The network therefore quickly acquires favorable pattern-selectivity by the mere repetitive presentation of a set of learning patterns.

This paper discusses color representation in the visual system by analysis of a three-layered neural network model. The model incorporates physiological knowledge of color representation at the sensor level (broad-band trichromatic representation by cones) and the higher level (narrow-band color representation by color-coded cells in V4). We trained the model to perform a mapping between these color representations by the back propagation algorithm and analyzed the acquired characteristics of the hidden units. It turned out that the hidden units learned characteristics similar to those of the color opponent cells found in the visual system. It was concluded that the R-G and Y-B color opponent representations reflect the efficiency of the color representation in the visual system from investigations on the efficiency of color representation in the hidden layer and on the capability of the color recognition task of the model.