Construction of a large‐scale neural network: Simulation of handwritten Japanese character recognition on NCUBE

Abstract

This paper describes how new learning methods may make it possible for a large‐scale, hierarchical neural network to recognize most Japanese handwritten characters. This is a very large and complex task, as the Japanese character set consists of about 3000 categories which can be written in many different ways. Such a difficult task can lead a neural network to converge very slowly and to yield recognition rates that are uneven between categories. To address these problems we here propose five learning methods as modifications of the conventional back‐propagation learning rule. These methods produce fast convergence, even recognition rates over all categories, and adequate recognition of test samples. We also describe how a large‐scale neural network can be built by dividing the recognition task into several subtasks, with networks for each subtask, and then integrating these subnetworks in a large network with a hierarchical structure. In a hierarchical network, the upper level network directly integrates outputs from each lower level network. Application of that network to handwritten Japanese character recognition has resulted in poor recognition, because lower level networks do not know about unknown input patterns, and the direct integration of ambiguous outputs from many lower level networks confuses the upper level network. We propose a new integration method which provides each subnetwork with more information as to how close an input pattern is to the categories of that subnetwork. This method resulted in high recognition performance for character recognition. We here described the above methods, and report the performance of our implementation of a neural network for the recognition of 71 Hiragana characters, and describe our implementation of this network on a hypercube concurrent computer.

12. The role of learning in theories of English and Japanese sentence processing

“…Processing in the brain's neural network is thought to involve spreading activation between neurons in ways that reflect the strength of their connections. In particular, these algorithms have been applied to various Japanese phenomena like kanji and phoneme recognition (Dominey, Hoen and Inui 2006;Ijuin et al 2000;Joe, Mori and Miyake 1990;Mori and Yokosawa 1989;Negishi 2006;Tsuzuki 1996;Waibel, Hanazawa, Hinton, Shikano and Lang, 1988). These models use artificial neuronlike units which have weights that represent the strength of their connection to other neurons, and these weights may be learned using different connectionist algorithms.…”

Fast modular neural nets for detection of human faces

“…Such tasks tend to introduce a wide range of overlap which, in turn, causes a wide range of deviations .from efficient learning in the different regions of input space [3]. High coupling among hidden nodes will then, result in over and under leaming at different regions [8]. Enlarging the network, increasing the number and quality of training samples, and techniques for avoiding local minina, will not stretch the learning capabilities of the NN classifier beyond a certain limit as long as hidden nodes are tightly coupled, and hence cross talking during leaming [7].…”

Fast co-operative modular neural processors for human face detection

“…Such tasks tend to introduce a wide range of overlap which, in turn, causes a wide range of deviations from efficient learning in the different regions of input space [3,5]. High coupling among hidden nodes will then, result in over and under learning at different regions [8]. Enlarging the network, increasing the number and quality of training samples, and techniques for avoiding local minina, will not stretch the learning capabilities of the NN classifier beyond a certain limit as long as hidden nodes are tightly coupled, and hence cross talking during learning [7].…”