A study of English word category prediction based on neural networks

Abstract

This paper describes word category prediction based on neural network models for constructing an accurate word recognition system. It is difficult to represent hidden linguistic structure and make an N-gram word prediction model using traditional stochastic approaches. In this paper, two neural network models that can learn hidden linguistic structure are proposed. These models can easily be expanded from Bigram to N-gram networks. They were tested by training experiments with an open English text database. The Trigram word category prediction rates show that neural network models are comparable to stochastic models. Trigram neural network models compress information about 150 times, which is the ratio of the Trigram stochastic model free parameters (893 = 704 969) to the neural network model link weights (4649). In addition, this paper proposes a new method that dynamically controls the training parameters, updating step size and momentum. These techniques are effective for calculating the efficiency of this system.

Recursive hashing functions for n -grams

“…When the symbols are words, n-grams have been exploited for word recognition in speech [Paeseler and Ney 1989;Pietra et al 1992;Wright et al 1992] and word categorization [Nakamura and Shikano 1989].…”

Speech Processing, Recognition and Artificial Neural Networks

“…Comparatively little work has been devoted to the very important field of language modelling. Some work concerns connectionist language modelling [113] and pronunciation dictionary generation [114].…”

Current Issues in Parsing Technology

“…Other work has focused on semantics and has not addressed the problem of grammar acquisition (Cottrell, 1985(Cottrell, , 1989. Connectionist networks have also been applied in building statistical language models for use in speech recognition (Nakamura & Shikano, 1989). However, such language models do not produce traditional parses of sentences; they seek to use statistical regularities in language structure and word usage directly for recognition tasks.…”