Fault tolerant constructive algorithm for feedforward neural networks

Hammadi, Nait Charif; Ohmameuda, Toshiaki; Kaneko, Kunihiko; Ito, Hideo

doi:10.1109/prfts.1997.640150

Cited by 23 publications

(29 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In (Hammadi, Ohmameunda, Kaneo, & Ito, 1998), a dynamic constructive algorithm is used to construct fault tolerant feedforward networks. This dynamic constructive fault tolerant algorithm (DCFTA) estimates a relevance factor for each weight and uses it to update the weights in a selective manner.…”

Section: Discussionmentioning

confidence: 99%

The constraint based decomposition (CBD) training architecture

Drăghici

2001

Neural Networks

View full text Add to dashboard Cite

The Constraint Based Decomposition (CBD) is a constructive neural network technique that builds a three or four layer neural network and is guaranteed to find a solution for any classification problem. CBD is shown to be able to solve complicated problems in a simple, fast and reliable manner. The technique is further enhanced by two modifications (locking detection and redundancy elimination) which address the training speed and the efficiency of the internal representation built by the network. The redundancy elimination aims at building more compact architectures while the locking detection aims at improving the training speed. The computational cost of the redundancy elimination is negligible and this enhancement can be used for any problem. However, the computational cost of the locking detection is exponential in the number of dimensions and should only be used in low dimensional spaces. The experimental results show the performance of the algorithm presented in a series of classical benchmark problems including the 2-spiral problem and the Iris, Wine, Glass, Lenses, Ionosphere, Lung cancer, Pima indians, Bupa, Tic-Tac-Toe, Balance and Zoo data sets from the CMU machine learning repository. CBD's generalization accuracy is compared with that of C4.5, C4.5 with rules, incremental decision trees, oblique classifiers, linear machine decision trees, CN2, learning vector quantization (LVQ), backpropagation, nearest neighbor, Q* and radial basis functions (RBFs). CBD provides the second best average accuracy on the problems tested as well as the best reliability (the lowest standard deviation).

show abstract

Section: Discussionmentioning

confidence: 99%

The constraint based decomposition (CBD) training architecture

Drăghici

2001

Neural Networks

View full text Add to dashboard Cite

show abstract

“…A method to improve the fault tolerance of backpropagation networks is presented in [30], which restrained the magnitudes of the connections during training process. Hammadi and Ito [16] demonstrate a training algorithm that reduces the relevance of weight. In [16], relevance of weight in each training epoch was estimated, and then decreases the magnitude of weight IV.…”

Section: Related Workmentioning

confidence: 99%

“…Hammadi and Ito [16] demonstrate a training algorithm that reduces the relevance of weight. In [16], relevance of weight in each training epoch was estimated, and then decreases the magnitude of weight IV.…”

Section: Related Workmentioning

confidence: 99%

Empirical Study of FFANN Tolerance to Weight Stuck at Max/Min Fault

Singh¹,

Shekhar²,

Chandra³

2010

IJAIA

View full text Add to dashboard Cite

Abstract-Fault tolerance property of artificial neural networks has been investigated with reference to the hardware model of artificial neural networks. Weight fault is an important link, which causes breakup between two nodes. In this paper weight fault has been explained. Experiments have been performed for Weight-stuck-0 fault. Effect of weight-stuck-0 fault on trained network has been analyzed in this paper. The obtained results suggest that networks are not fault tolerant to this type of fault.

show abstract

“…Improve tolerance of a neural network towards random node fault, stuck-at node fault and weight noise have been researching for almost two decades [4,6,5,7,9,12,13,16,17,19], Many methods such as injecting random node fault [18,3], injecting weight noise during training (for multilayer perceptrons (MLP) [14,15], a recurrent neural network (RNN) [11], or a pulse-coupled neural networks (PCNN) [8]) or node noise (response variability) during training [2] (for PCNN) during training have been developed and demonstrated with success via intensive computer simulations. Despite the idea of injecting weight noise during training is straight forward and its implementation is extremely elegant, theoretical analysis regrading their convergence and the objective functions in which the algorithms are minimizing is scarce [1,2,14,15].…”

Section: Introductionmentioning

confidence: 99%

SNIWD: Simultaneous Weight Noise Injection with Weight Decay for MLP Training

Sum

2009

Neural Information Processing

View full text Add to dashboard Cite

Abstract. Despite noise injecting during training has been demonstrated with success in enhancing the fault tolerance of neural network, theoretical analysis on the dynamic of this noise injection-based online learning algorithm has far from complete. In particular, the convergence proofs for those algorithms have not been shown. In this regards, this paper presents an empirical study on the non-convergence properties of injecting weight noises during training a multilayer perceptron, and an online learning algorithm called SNIWD (simultaneous noise injection and weight decay) to overcome such non-convergence problem. Simulation results show that SNIWD is able to improve the convergence and enforce small magnitude on the network parameters (input weights, input biases and output weights). Moreover, SNIWD is able to make the network have similar fault tolerance ability as using pure noise injection approach.

show abstract

Fault tolerant constructive algorithm for feedforward neural networks

Cited by 23 publications

References 10 publications

The constraint based decomposition (CBD) training architecture

The constraint based decomposition (CBD) training architecture

Empirical Study of FFANN Tolerance to Weight Stuck at Max/Min Fault

SNIWD: Simultaneous Weight Noise Injection with Weight Decay for MLP Training

Contact Info

Product

Resources

About