Limits to the fault-tolerance of a feedforward neural network with learning

Nijhuis, J.A.G.; Höfflinger, B.; Schaik, André van; Spaanenburg, Lambert

doi:10.1109/ftcs.1990.89370

Cited by 33 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The overtraining effect had been observed, but not explained, in studies of forgetting [2] and faulttolerance [23]. Avoidance of forgetting and increasing fault-tolerance can be seen as intimately related goals.…”

Section: B How To Prepare a Network For Damage Or The Relation Of Lmmentioning

confidence: 99%

Learning of nonstationary processes

Angulo¹,

Torras²

1998

Optimization Techniques

View full text Add to dashboard Cite

If a very different and representative input-output pattern needs to be learned after the training of the main set of patterns has been completed, one gets into trouble. One can train the new pattern isolatedly or retrain a mixture of the new and old patterns. The first option is the best for this kind of applications because a quicker adaptation is obtained, but the interference must not be very strong, or at least one must be able to mitigate it. Unfortunately, the forgetting problem turns out to be a very serious drawback of back-propagation networks.Although some interest has recently emerged on this issue, the connectionist community has not yet paid enough attention to it. Ratcliff [1] and Mc Closkey & Cohen [2], after many systematic studies, simply arrive to the conclusion that this problem cannot be satisfactorily solved. French [3] claims that the cause of forgetting is the overlap between the representations of the different patterns and, therefore, he modifies back-propagation so as to produce semi-distributed representations. In these representations, only a few hidden units take the value 1, while the majority take the value 0. This type of approach has very serious convergence problems and requires a much larger number of hidden units than straight back-propagation [4]. Reducing the distributedness of the representations has also the very undesirable effect of losing some of the most interesting neural network properties, like generalization (as French himself points out) and damage resistance. happens also in some of Hetherington's own experiments regarding the influence of increasing the training set size. Sharkey and Sharkey (1994) used also autocoder networks in their studies of catastrophic interference. We think that results about forgetting obtained with autocoder networks and low-error patterns must not be extrapolated to more general situations. Krusche [9] has pointed out that the huge receptive field of the weighted-sum units is responsible for interference in neural networks. Units with a limited receptive field are increasingly being used [10]

show abstract

Section: B How To Prepare a Network For Damage Or The Relation Of Lmmentioning

confidence: 99%

Learning of nonstationary processes

Angulo¹,

Torras²

1998

Optimization Techniques

View full text Add to dashboard Cite

show abstract

“…This increases the probability, to a level which is not negligible, that a fault will develop at one of the components. Even though the neural network has an inherent fault tolerance due to its redundancy, some faulttolerant mechanism must be incorporated so that the neural network hardware can be realized with a sufficient reliability [13,14].…”

Section: Introductionmentioning

confidence: 99%

Fault-tolerant neural networks with higher functionality

Tohma

Iwata

1999

Syst. Comp. Jpn.

View full text Add to dashboard Cite

It is known that mutually coupled neural networks are suited for solving optimization problems. So far, we have shown that a neural network which is tolerant to unidirectional faults can be realized by selecting automatically one of the two complementary representations of the solution. It is also shown that a fault‐tolerant neural network can be realized by a triplication structure with an appropriate merging function, even if both stuck‐at‐1 and 0 faults may occur simultaneously. In these realizations, the conventional property of a neuron, a component of the neural network, is utilized, where the output of the neuron is determined by the sigmoid function of the sum of its inputs multiplied by the respective weights (connection coefficients). This paper shows that if a neuron can have a higher function than the weighted sum of its inputs, a neural network which is tolerant to stuck‐at‐1 and 0 mixed faults can be realized by the duplication structure. Since the duplication structure suffices, it is expected that less hardware will be needed than in the triplication structure. Although the higher function of the neuron assumed in this paper has not actually been realized, we expect that such a function will be realized in hardware in the near future as a result of progress in integrated circuit technology. In this sense, the proposal in this paper is practically realistic, and provides a new means of implementing fault tolerance of neural networks. © 1999 Scripta Technica, Syst Comp Jpn, 30(10): 22–33, 1999

show abstract

“…Noise injection into the training set has been reported in [9] in the case of a digital approach. In our case, using a noisy training set is intended to mimic the imperfect behavior or effect of a damaged or leaky device.…”

Section: Research Perspectivesmentioning

confidence: 99%

“…The algorithmic aspects, as well as the hardware implementation issues have been addressed, mainly with the target of guaranteeing the ANN transfer function under failure, while considering a minimal number of neurons [8][9]. Also the aspects of learning using error-prone analog hardware and/or limited precision digital circuits have been addressed.…”

Section: Introductionmentioning

confidence: 99%

A modular approach for reliable nanoelectronic and very-deep submicron circuit design based on analog neural network principles

Schmid

Leblebici

2003 Third IEEE Conference on Nanotechnology, 2003. IEEE-NANO 2003.

View full text Add to dashboard Cite

Abstract-Reliability of nanodevices is expected to be a central issue with the advent of very-deep submicon devices and future single-electron transistors. We propose a new approach based on the assumption that a number of circuit-level devices are to be expected to fail. Artificial neural networks can be trained to resists to errors and be used for synthesizing fault-tolerant Boolean functions. The development method is outlined; results based on the feed-forward artificial neural network implementation are presented, while future research directions are discussed with possible applications. Fault-tolerance; robust very-deep submicron design; artificial neural networks

show abstract

Limits to the fault-tolerance of a feedforward neural network with learning

Cited by 33 publications

References 8 publications

Learning of nonstationary processes

Learning of nonstationary processes

Fault-tolerant neural networks with higher functionality

A modular approach for reliable nanoelectronic and very-deep submicron circuit design based on analog neural network principles

Contact Info

Product

Resources

About