Reducing Network Depth in the Cascade-Correlation Learning Architecture,

Baluja, Shumeet; Fahlman, Scott E.

doi:10.21236/ada289352

Cited by 80 publications

(75 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In cascade correlation neural networks training starts with a simple perceptron network, which is grown incrementally by adding new cascaded layers with skip-level connections as long as performance on a validation dataset improves. Since the proposal of the original cascade correlation algorithm in [7], various improvements that follow a similar overall process to the original method have been proposed, for example in [1,11,20,31], as well as Cascade2 [21]. Active research in this field, however, is fairly limited.…”

Section: Automlmentioning

confidence: 99%

CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification (Best Technical Paper)

Pakrashi

Namee

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multi-label classification is an approach which allows a datapoint to be labelled with more than one class at the same time. A common but trivial approach is to train individual binary classifiers per label, but the performance can be improved by considering associations within the labels. Like with any machine learning algorithm, hyperparameter tuning is important to train a good multi-label classifier model. The task of selecting the best hyperparameter settings for an algorithm is an optimisation problem. Very limited work has been done on automatic hyperparameter tuning and AutoML in the multi-label domain. This paper attempts to fill this gap by proposing a neural network algorithm, CascadeML, to train multi-label neural network based on cascade neural networks. This method requires minimal or no hyperparameter tuning and also considers pairwise label associations. The cascade algorithm grows the network architecture incrementally in a two phase process as it learns the weights using adaptive first order gradient algorithm, therefore omitting the requirement of preselecting the number of hidden layers, nodes and the learning rate. The method was tested on 10 multi-label datasets and compared with other multi-label classification algorithms. Results show that CascadeML performs very well without hyperparameter tuning.

show abstract

Section: Automlmentioning

confidence: 99%

CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification (Best Technical Paper)

Pakrashi

Namee

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Instead of BP, we use a variant of the cascade correlation (CC) method called sibling-descendant cascade correlation (SDCC) which is a constructive method for learning in multi-layer artificial neural networks (Baluja & Fahlman, 1994). SDCC learns both the network's structure and the connection weights; it starts with a minimal network, then automatically trains new hidden units and adds them to the active network, one at a time.…”

Section: Autonomous Learning Via a Constructive Algorithmmentioning

confidence: 99%

“…In classical CC, each new recruit is installed on its own layer, higher than previous layers. The SDCC variant is more flexible in that a recruit can be installed either on the current highest layer (as a sibling) or on its own higher layer as a descendant, depending on which location yields the higher correlation between candidate unit activation and current network error (Baluja & Fahlman, 1994). In both CC and SDCC, learning progresses in a recurring sequence of two phases -output phase and input phase.…”

Section: Autonomous Learning Via a Constructive Algorithmmentioning

confidence: 99%

Neural implementation of probabilistic models of cognition

Kharratzadeh

Shultz

2016

Cognitive Systems Research

View full text Add to dashboard Cite

Bayesian models of cognition hypothesize that human brains make sense of data by representing probability distributions and applying Bayes' rule to find the best explanation for available data. Understanding the neural mechanisms underlying probabilistic models remains important because Bayesian models provide a computational framework, rather than specifying mechanistic processes. Here, we propose a deterministic neural-network model which estimates and represents probability distributions from observable events -a phenomenon related to the concept of probability matching. Our model learns to represent probabilities without receiving any representation of them from the external world, but rather by experiencing the occurrence patterns of individual events. Our neural implementation of probability matching is paired with a neural module applying Bayes' rule, forming a comprehensive neural scheme to simulate human Bayesian learning and inference. Our model also provides novel explanations of base-rate neglect, a notable deviation from Bayes.

show abstract

“…It is possible that the random functions generated by our teacher networks did not have this characteristic. Baluja and Fahlman (1994) proposed an elegant solution to the problem of having to choose whether or not to cascade weights. In their extension to cascor, called Sibling/Descendant Downloaded by [Bangor University] at 19:25 03 January 2015…”

Section: Why Use Cascading Weights?mentioning

confidence: 99%

A systematic comparison of flat and standard cascade-correlation using a student–teacher network approximation task

2007

View full text Add to dashboard Cite

Cascade-correlation (cascor) networks grow by recruiting hidden units to adjust their computational power to the task being learned. The standard cascor algorithm recruits each hidden unit on a new layer, creating deep networks. In contrast, the flat cascor variant adds all recruited hidden units on a single hidden layer. Student-teacher network approximation tasks were used to investigate the ability of flat and standard cascor networks to learn the input-output mapping of other, randomly initialized flat and standard cascor networks. For low-complexity approximation tasks, there was no significant performance difference between flat and standard student networks. Contrary to the common belief that standard cascor does not generalize well due to cascading weights creating deep networks, we found that both standard and flat cascor generalized well on problems of varying complexity. On high-complexity tasks, flat cascor networks had fewer connection weights and learned with less computational cost than standard networks did.

show abstract

Reducing Network Depth in the Cascade-Correlation Learning Architecture,

Cited by 80 publications

References 5 publications

CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification (Best Technical Paper)

CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification (Best Technical Paper)

Neural implementation of probabilistic models of cognition

A systematic comparison of flat and standard cascade-correlation using a student–teacher network approximation task

Contact Info

Product

Resources

About