The negative transfer problem in neural networks: a solution

Abunawass, Adel M.

doi:10.1109/ijcnn.1991.170511

Cited by 4 publications

(13 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second important observation here is that the range of synaptic weight distribution widened with the increase in the amount of training. As implied in [2], the range of weight distribution is an important factor that induces the negative transfer problem.…”

Section: Synaptic Weight Distribution Andmentioning

confidence: 99%

“…The results of this experiment are shown in Figure 2. Modification in equation (1) was proposed by Abunawass [2], however, in [2], this computation takes place at every epoch of training.…”

Section: Wnew(i) = ( W O L D ( I ) -P O L D ) / ( B O L D ) (1)mentioning

confidence: 99%

“…For software simulations this solution is practical since storing of weights is a common practice to observe the network's behavior for different tasks. Besides to store the weight file at early trained stage, if additional weight storage is indeed a problem, tbe weights could be modified using the technique mentioned in [2]. For hardware realization, additional circuitry is required to perform the range limitation and normalization.…”

Section: Negative Transfer Problem and Fault Tolerancementioning

confidence: 99%

“…That is, training a trained bp neural network for new training set can be extremely difficult. This has been defined by Abunawass as the negative transfer problem of bp algorithm [4]. A solution has been proposed also by Abunawass [2], which modifies the bp algorithm such that the synaptic weights are limited after each epoch of training.…”

Section: Introductionmentioning

confidence: 99%

“…The VLSI realization of the modified bp algG rithm proposed in [2] is fairly complex. Firstly, the synaptic weights are transformed using the mean and the variance of all synaptic weights.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A suggestion for solving negative transfer problem in VLSI backpropagation neural networks

Kolluri

Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)

View full text Add to dashboard Cite

show abstract

Section: Synaptic Weight Distribution Andmentioning

confidence: 99%

“…The results of this experiment are shown in Figure 2. Modification in equation (1) was proposed by Abunawass [2], however, in [2], this computation takes place at every epoch of training.…”

Section: Wnew(i) = ( W O L D ( I ) -P O L D ) / ( B O L D ) (1)mentioning

confidence: 99%

Section: Negative Transfer Problem and Fault Tolerancementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A suggestion for solving negative transfer problem in VLSI backpropagation neural networks

Kolluri

Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94)

View full text Add to dashboard Cite

show abstract

Parallel processing of chemical information in a local area network—II. A parallel cross-validation procedure for artificial neural networks

Derks

Beckers

Melssen

et al. 1996

Computers & Chemistry

View full text Add to dashboard Cite

<title>Negative transfer problem in neural networks</title>

Abunawass

1992

SPIE Proceedings

View full text Add to dashboard Cite

Harlow, 1949, observed that when human subjects were trained to perform simple discrimination tasks over a sequence of successive training sessions (trials), their performance improved as a function of the successive sessions. Harlow called this phenomena "learning-to-learn" . The subjects acquired knowledge and improved their ability to learn in future training sessions. It seems that previous training sessions contribute positively to the current one. , observed that when a neural network (using the back-propagation model) is trained over successive sessions, the performance and learning ability of the network degrade as a function of the training sessions. In some cases this leads to a complete paralysis of the network. Abunawass & Maki called this phenomena the "negative transfer" problem, since previous training sessions contribute negatively to the current one. The effect of the negative transfer problem is in clear contradiction to that reported by Harlow on human subjects. Since the ability to model human cognition and learning is one of the most important goals (and claims) of neural networks, the negative transfer problem represents a clear limitation to this ability. This paper describes a new neural network sequential learning model known as Adaptive Memory Consolidation. In this model the network uses its past learning experience to enhance its future learning ability. Adaptive Memory Consolidation has led to the elimination and reversal of the effect of the negative transfer problem. Thus producing a "positive transfer" effect similar to Harlow's learning-to-learn phenomena. BACKGROUND Artificial Neural NetworksArtificial neural networks (also known as connectionist models, adaptive neural networks, neural networks, adaptive systems, neuromorphic systems and parallel distributed processing) have a rich history, numerous successes and a wide appeal. The appeal of artificial neural networks includes the potential of brain-like computations and human-like learning and intelligence. We believe that for an artificial model to be a viable model of learning and intelligence, the model must be capable of matching the experimental results of its natural counterpart. Loosely stated, the artificial model must be able to mimic and exhibit the behavior of the natural one. Artificial neural networks have problems that limit their ability to mimic the behavior of the natural ones; one such problem is the topic of this paper. In the paper we are limiting our focus to the back-propagation (BP) model [8].Artificial neural networks have a similar topography to directed graphs (see Figure 1). The nodes in the graph are known as units or neurons. The edges of the graph are known as connections or synapses. The units are linked to each other via connections. The connections are unidirectional. Each unit has output and input values. Each connection has a weight value. The weights of the connections represent the long term memory (LTM) of the network. LTM contains the learning the network has accomplished. LTM is the perma...

show abstract

The negative transfer problem in neural networks: a solution

Cited by 4 publications

References 5 publications

A suggestion for solving negative transfer problem in VLSI backpropagation neural networks

A suggestion for solving negative transfer problem in VLSI backpropagation neural networks

Parallel processing of chemical information in a local area network—II. A parallel cross-validation procedure for artificial neural networks

<title>Negative transfer problem in neural networks</title>

Contact Info

Product

Resources

About