Learning System Architectures Composed of Multiple Learning Modules

When networks disagree: Ensemble methods for hybrid neural networks

“…Of course this approach will be prone to local minima and noise due to stochastic gradient descent just as the original population Y was; thus we can train a population of perceptrons to combine the networks from Y and then average over this new population. A further extension is to use a nonlinear network (Jacobs et al, 1991;Reilly et al, 1987;Wolpert, 1990) to learn how to combine the networks with weights that vary over the feature space and then to average an ensemble of such networks. This extension is reasonable since networks will in general perform better in certain regions of the feature space than in others.…”

“…Hybrid or multi-neural network systems have been frequently employed to improve results in classification and regression problems (Cooper, 1991;Reilly et al, 1988;Reilly et al, 1987;Scofield et al, 1991;Baxt, 1992;Bridle and Cox, 1991;Buntine and Weigend, 1992;Hansen and Salamon, 1990;Intrator et al, 1992;Jacobs et al, 1991;Lincoln and Skrzypek, 1990;Neal, 1992a;Neal, 1992b;Pearlmutter and Rosenfeld, 1991;Wolpert, 1990;Xu et al, 1992;Xu et al, 1990). Among the key issues are how to design the architecture of the networks; how the results of the various networks should be combined to give the best estimate of the optimal result; and how to make 1 best use of a limited data set.…”

Computational concepts in classification: Neural networks, statistical pattern recognition, and model-based vision

“…Various algorithms differ in the way they prune and retain the exemplars. This approach is used by the RCE and PNN neural networks [13], [14]. These algorithms can be used for both supervised and unsupervised classification.…”

A novel recursive partitioning criterion

“…These results represent worst case bounds for network performance and can be improved by using data-driven techniques if one can assume the existence of appropriate structure in the data. (See, for example, Bachmann90, Breiman84, Ersoy90, Friedman88, Nowlan90,Reilly88,Reilly87,Sanger90,Sankar9l) One can use hybrid networks to implement data-driven algorithms in a neural network setting. (See Cooper9l for further discussion and references.)…”