Gradient descent fails to separate

Brady, Michael; Raghavan, Vasudevan; Slawny,

doi:10.1109/icnn.1988.23902

Cited by 12 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is well known from the perceptron convergence theorem [2]- [4] that the weights of the perceptron will converge to a fixed point within a finite number of updates if the set of bounded training feature vectors is linearly separable, and the weights may exhibit a limit cycle behavior if the set of bounded training feature vectors is nonlinearly separable. However, the exact condition for the weights of the perceptron exhibiting the limit cycle behavior is unknown.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Global Convergence and Limit Cycle Behavior of Weights of Perceptron

Ling

Lam

et al. 2008

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

In this paper, it is found that the weights of a perceptron are bounded for all initial weights if there exists a nonempty set of initial weights that the weights of the perceptron are bounded. Hence, the boundedness condition of the weights of the perceptron is independent of the initial weights. Also, a necessary and sufficient condition for the weights of the perceptron exhibiting a limit cycle behavior is derived. The range of the number of updates for the weights of the perceptron required to reach the limit cycle is estimated. Finally, it is suggested that the perceptron exhibiting the limit cycle behavior can be employed for solving a recognition problem when downsampled sets of bounded training feature vectors are linearly separable. Numerical computer simulation results show that the perceptron exhibiting the limit cycle behavior can achieve a better recognition performance compared to a multilayer perceptron.

show abstract

Section: Introductionmentioning

confidence: 99%

“…B ECAUSE the implementation cost of a perceptron is low and a perceptron can classify linearly separable bounded training feature vectors [2]- [4], perceptrons [6]- [10] are widely applied in many pattern recognition systems.…”

Section: Introductionmentioning

confidence: 99%

Global Convergence and Limit Cycle Behavior of Weights of Perceptron

Ling

Lam

et al. 2008

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

show abstract

“…This allows us to use the mean value theorem to bound d k ?y k W T X k ?W T k X k . In the following section we will consider an alternative convergence 4 The standard inequality is written as…”

mentioning

confidence: 99%

Adaptive receiver algorithms for near-far resistant CDMA

Mitra

Poor

1995

IEEE Trans. Commun.

View full text Add to dashboard Cite

show abstract

“…However, some constructive algorithms have a very limited scope and in general, they are believed to give poor generalization results. Non-constructive algorithms are much less reliable than constructive algorithms and can fail to converge even when a solution exists (Brady, Raghavan, & Slawny, 1988). Some non-constructive algorithms can even converge to 'false' solutions (Brady, Raghavan, & Slawny, 1989).…”

Section: Introductionmentioning

confidence: 99%

The constraint based decomposition (CBD) training architecture

Drăghici

2001

Neural Networks

View full text Add to dashboard Cite

The Constraint Based Decomposition (CBD) is a constructive neural network technique that builds a three or four layer neural network and is guaranteed to find a solution for any classification problem. CBD is shown to be able to solve complicated problems in a simple, fast and reliable manner. The technique is further enhanced by two modifications (locking detection and redundancy elimination) which address the training speed and the efficiency of the internal representation built by the network. The redundancy elimination aims at building more compact architectures while the locking detection aims at improving the training speed. The computational cost of the redundancy elimination is negligible and this enhancement can be used for any problem. However, the computational cost of the locking detection is exponential in the number of dimensions and should only be used in low dimensional spaces. The experimental results show the performance of the algorithm presented in a series of classical benchmark problems including the 2-spiral problem and the Iris, Wine, Glass, Lenses, Ionosphere, Lung cancer, Pima indians, Bupa, Tic-Tac-Toe, Balance and Zoo data sets from the CMU machine learning repository. CBD's generalization accuracy is compared with that of C4.5, C4.5 with rules, incremental decision trees, oblique classifiers, linear machine decision trees, CN2, learning vector quantization (LVQ), backpropagation, nearest neighbor, Q* and radial basis functions (RBFs). CBD provides the second best average accuracy on the problems tested as well as the best reliability (the lowest standard deviation).

show abstract

Gradient descent fails to separate

Cited by 12 publications

References 1 publication

Global Convergence and Limit Cycle Behavior of Weights of Perceptron

Global Convergence and Limit Cycle Behavior of Weights of Perceptron

Adaptive receiver algorithms for near-far resistant CDMA

The constraint based decomposition (CBD) training architecture

Contact Info

Product

Resources

About