CARVE-a constructive algorithm for real-valued examples

Abstract

A constructive neural-network algorithm is presented. For any consistent classification task on real-valued training vectors, the algorithm constructs a feedforward network with a single hidden layer of threshold units which implements the task. The algorithm, which we call CARVE, extends the "sequential learning" algorithm of Marchand et al. from Boolean inputs to the real-valued input case, and uses convex hull methods for the determination of the network weights. The algorithm is an efficient training scheme for producing near-minimal network solutions for arbitrary classification tasks. The algorithm is applied to a number of benchmark problems including Gorman and Sejnowski's sonar data, the Monks problems and Fisher's iris data. A significant application of the constructive algorithm is in providing an initial network topology and initial weights for other neural-network training schemes and this is demonstrated by application to backpropagation.

A convergent algorithm for a cascade network of multiplexed dual outputdiscrete perceptrons for linearly nonseparable classification

“…when it is less than about 75%, the PLR leads to the best or the second best generalization performance. However, there is no possibility of direct comparison of the proposed method to the previous works in terms of the generalization performance, since the most similar methods, those also employing the same data sets [4,5,22,23], do not declare their generalization performances. The generalization performance of the proposed cascade network is discussed in Section 4.5.…”

“…However, the work in [4] dealt with Boolean inputs only. Another algorithm, based on SLA, is the constructive algorithm for real-valued examples (CARVE) [5], which extends the SLA from * Correspondence: ibrahim.genc@medeniyet.edu.tr Boolean inputs to real-valued input cases; it uses a convex hull method for the determination of the network weights instead of the PLR. This algorithm gives a near-optimal solution since the task of finding the largest appropriate set is NP-hard and the algorithm only finds good-sized appropriate sets.…”

“…These models form a cascade structure where the inputs [6,7] or the bias-inputs [8] of higher layers' neurons are fed by the outputs of the lower layers' neurons. These models propose structural and algorithmic solutions to linearly nonseparable classification problems against the ones [4,5] that propose only algorithmic solutions.…”

Binary classification by SVM based tree type neural networks

“…Samples belonging to Class 1 are located at the points with co-ordinates (1,8), (4,5), (4,4), (1,1), (6,5), (6,4), (10,8), (10,1), while samples belonging to Class 0 are located at the points with co-ordinates (2,6), (2,3), (8,6), (8,3). Figure 2 shows the SVM based tree type perceptron network having 6 nodes that learns the data.…”

“…Constructive methods begin with a small network and gradually increase its size when necessary, in order to complete the learning task. Some constructive techniques for developing neural network architectures can be found in [3][4][5][6][7]. The basic objective in these techniques is to reduce the complexity of the model and also to satisfy some performance criterion.…”