Backpropagation separates when perceptrons do

Sontag,; Sussmann,

doi:10.1109/ijcnn.1989.118644

Cited by 21 publications

(6 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the target for a pattern of class 1 is 1 -f but the output is greater than 1 -E, then the output is clamped to 1 -f. Similarly for a pattern of class 2, if the target is td but the network output is less than f, then the output is clamped to f. The clamp is used to implement the modified penalty function suggested by Sontag and Sussmann [11]. They observe that backpropagation is less likely to get stuck in local minima when the output is clamped during training.…”

Section: Definitionsmentioning

confidence: 99%

An improved algorithm for neural network classification of imbalanced training sets

Anand

Mehrotra

Mohan

et al. 1993

IEEE Trans. Neural Netw.

227

108

View full text Add to dashboard Cite

In this paper, we analyze the reason for the slow rate of convergence of net output error when using the backpropagation algorithm to train neural networks for a two-class problems in which the numbers of exemplars for the two classes differ greatly. This occurs because the negative gradient vector computed by backpropagation for an imbalanced training set does not point initially in a downhill direction for the class with the smaller number of exemplars. Consequently, in the initial iteration, the net error for the exemplars in this class increases significantly. The subsequent rate of convergence of the net error is very low. We suggest a modified technique for calculating a direction in weight-space which is downhill for both classes. Using this algorithm, we have been able to accelerate the rate of learning for two-class classification problems by an order of magnitude.

show abstract

Section: Definitionsmentioning

confidence: 99%

An improved algorithm for neural network classification of imbalanced training sets

Anand

Mehrotra

Mohan

et al. 1993

IEEE Trans. Neural Netw.

227

108

View full text Add to dashboard Cite

show abstract

“…This threshold-LMS error has been introduced by Sontag & Sussman (1989). This cost does not penalize outputs "beyond" the target values.…”

Section: -(O(t(q))-d-)mentioning

confidence: 99%

“…The target values were 0.9 for grammatical and 0.1 for ungrammatical strings. A LMS-threshold function was used for the error computation (Sontag & Sussman, 1989).…”

Section: Experimental Set Upmentioning

confidence: 99%

Representation of finite state automata in Recurrent Radial Basis Function networks

1996

View full text Add to dashboard Cite

Abstract. In this paper, we propose some techniques for injecting finite state automata into Recurrent Radial Basis Function networks (R2BF). When providing proper hints and constraining the weight space properly, we show that these networks behave as automata. A technique is suggested for forcing the learning process to develop automata representations that is based on adding a proper penalty function to the ordinary cost. Successful experimental results are shown for inductive inference of regular grammars.

show abstract

“…We remark that an error function similar to (5) has been considered by Sontag and Sussmann in [10], where, with reference to multilayer perceptron networks, the properties of its local minima have been analyzed. However, as far as we are aware, the use of such an error function for addressing the problem of network training has not yet been proposed.…”

Section: An Alternative Formulation Of the Training Problem For Pmentioning

confidence: 99%

Efficient training of RBF neural networks for pattern recognition

Lampariello

Sciandrone

2001

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

Abstract-The problem of training a radial basis function (RBF) neural network for distinguishing two disjoint sets in is considered. The network parameters can be determined by minimizing an error function that measures the degree of success in the recognition of a given number of training patterns. In this paper, taking into account the specific feature of classification problems, where the goal is to obtain that the network outputs take values above or below a fixed threshold, we propose an approach alternative to the classical one that makes use of the least-squares error function. In particular, the problem is formulated in terms of a system of nonlinear inequalities, and a suitable error function, which depends only on the violated inequalities, is defined. Then, a training algorithm based on this formulation is presented. Finally, the results obtained by applying the algorithm to two test problems are compared with those derived by adopting the commonly used leastsquares error function. The results show the effectiveness of the proposed approach in RBF network training for pattern recognition, mainly in terms of computational time saving.Index Terms-Error functions, neural-network training, pattern recognition.

show abstract

Backpropagation separates when perceptrons do

Cited by 21 publications

References 3 publications

An improved algorithm for neural network classification of imbalanced training sets

An improved algorithm for neural network classification of imbalanced training sets

Representation of finite state automata in Recurrent Radial Basis Function networks

Efficient training of RBF neural networks for pattern recognition

Contact Info

Product

Resources

About