Self-organizing network for optimum supervised learning

Tenorio, M.F.; Lee, W.-T.

doi:10.1109/72.80209

Cited by 86 publications

(27 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since these specification terms also depend on the particular coding scheme (or the prior bias in Bayesian terminology), we choose to ignore these terms in the optimization or search and propose that the complexity be simply proportional to the (3/2) log N precision terms. This penalty scheme has been widely used in practice by other authors (Rissanen and Wax 1988;Tenorio and Lee 1990, et al). Hence, for rule set R the complexity is assessed as As we shall discuss later in the section on empirical results, this simple pruning algorithm is very effective at discovering parsimonious rule sets that account for the data.…”

Section: Rule Pruning Using a Minimum Description Length Model ~mentioning

confidence: 99%

Rule-Based Neural Networks for Classification and Probability Estimation

et al. 1992

View full text Add to dashboard Cite

In this paper we propose a network architecture that combines a rulebased approach with that of the neural network paradigm. Our primary motivation for this is to ensure that'the knowledge embodied in the network is explicitly e'ncoded in-the form of understandable rules. This enables the network's decision to be understood, and provides an audit trail of how that decision was arrived at. We utilize an information theoretic approach to leafning a model of the domain knowledge from examples. This model takes ;he form of a set of probabilistic conjunctive rules between discrete input evidence variables and output class variables. These rules are then mapped onto the weights and nodes of a feedforward neural network resulting in a directly specified architecture. The network acts as parallel Bayesian classifier, but more importantly, can also output posterior probability estimates of the class variables. Empirical tests on a number of data sets show that the rulebased classifier performs comparably with standard neural network classifiers, while possessing unique advantages in terms of knowledge representation and probability estimation.

show abstract

Section: Rule Pruning Using a Minimum Description Length Model ~mentioning

confidence: 99%

Rule-Based Neural Networks for Classification and Probability Estimation

et al. 1992

View full text Add to dashboard Cite

show abstract

“…Several constructive and growing algorithms have been proposed to address this issue, since they are capable of searching the nondifferentiable landscape of subsets of polynomial features. Some add units by multiplying extant ones with new inputs [13], and many others focus on the construction of multilayer FLNs [14]. A recent constructive method [15] performs a Boolean approximation of the data to select the relevant polynomials, followed by a final pruning phase.…”

Section: Efln In Contextmentioning

confidence: 99%

Evolution of functional link networks

Sierra

Macías

Corbacho

2001

IEEE Trans. Evol. Computat.

View full text Add to dashboard Cite

Abstract-This paper addresses the genetic design of functional link networks (FLNs). FLNs are high-order perceptrons (HOPs) without hidden units. Despite their linear nature, FLNs can capture nonlinear input-output relationships, provided that they are fed with an adequate set of polynomial inputs, which are constructed out of the original input attributes. Given this set, it turns out to be very simple to train the network, as compared with a multilayer perceptron (MLP). However, finding the optimal subset of units is a difficult problem because of its nongradient nature and the large number of available units, especially for high degrees. Some constructive growing methods have been proposed to address this issue. Here, we rely on the global search capabilities of a genetic algorithm to scan the space of subsets of polynomial units, which is plagued by a host of local minima. By contrast, the quadratic error function of each individual FLN has only one minimum, which makes fitness evaluation practically noiseless. We find that surprisingly simple FLNs compare favorably with other more complex architectures derived by means of constructive and evolutionary algorithms on some UCI benchmark data sets. Moreover, our models are especially amenable to interpretation, due to an incremental approach that penalizes complex architectures and starts with a pool of single-attribute FLNs.Index Terms-Evolutionary neural networks, feature subset selection, functional link networks, polynomial regression.

show abstract

“…Sometimes, "pruning" is also performed on nodes in the input and output layers in order to determine the most important set of variables in the representation scheme. The former approach is represented in the Dynamic Node Creation scheme [18], the Cascade Correlation Learning Architecture [13], and the Self-Organising Neural Network [14], and the latter in Skeletonization [19], and Kamin's pruning scheme [20].…”

Section: Higher Order Learning and Adaptive Architecture Determinationmentioning

confidence: 99%