Cumulative negative transfer during successive training: analysis of a second sequential learning problem

Abunawass,; Maki,

doi:10.1109/ijcnn.1989.118508

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper the existence of a second sequential learning problem and its effect on the performance of neural networks reported earlier in [2] was confirmed. The NT caused a considerable degradation in the network's performance regardless of its architecture.…”

Section: Discussionsupporting

confidence: 70%

“…Additionally, the mean and the median of the weight values remained relatively constant (and near zero) over the lists for both models (see [1,2] for details). For the OBP model, the standard of deviation, minimum and maximum values increased as a function of successive lists.…”

Section: Analysis Of the Weightsmentioning

confidence: 91%

“…A relative frequency distribution of the weights was then constructed. In the case of the OBP model (see [1,2] for details), Figure 4 shows that after training on list 1 the majority of the weights remained relatively small. The first bin has the range of [O,X); the second bin has the range of [X,2X); the third bin has the range of [2X,3X); .. .…”

Section: Analysis Of the Weightsmentioning

confidence: 99%

“…The error value for the output units is calculated as follows: api = ( tpj Opj ) f'(Ne,) (2) where f' (Net,,) = O ( 1 -), substituting in Eq. In this step the values of the output units are compared to the desired output (the teaching).…”

mentioning

confidence: 99%

See 3 more Smart Citations

<title>Negative transfer problem in neural networks</title>

Abunawass

1992

SPIE Proceedings

View full text Add to dashboard Cite

Harlow, 1949, observed that when human subjects were trained to perform simple discrimination tasks over a sequence of successive training sessions (trials), their performance improved as a function of the successive sessions. Harlow called this phenomena "learning-to-learn" . The subjects acquired knowledge and improved their ability to learn in future training sessions. It seems that previous training sessions contribute positively to the current one. , observed that when a neural network (using the back-propagation model) is trained over successive sessions, the performance and learning ability of the network degrade as a function of the training sessions. In some cases this leads to a complete paralysis of the network. Abunawass & Maki called this phenomena the "negative transfer" problem, since previous training sessions contribute negatively to the current one. The effect of the negative transfer problem is in clear contradiction to that reported by Harlow on human subjects. Since the ability to model human cognition and learning is one of the most important goals (and claims) of neural networks, the negative transfer problem represents a clear limitation to this ability. This paper describes a new neural network sequential learning model known as Adaptive Memory Consolidation. In this model the network uses its past learning experience to enhance its future learning ability. Adaptive Memory Consolidation has led to the elimination and reversal of the effect of the negative transfer problem. Thus producing a "positive transfer" effect similar to Harlow's learning-to-learn phenomena. BACKGROUND Artificial Neural NetworksArtificial neural networks (also known as connectionist models, adaptive neural networks, neural networks, adaptive systems, neuromorphic systems and parallel distributed processing) have a rich history, numerous successes and a wide appeal. The appeal of artificial neural networks includes the potential of brain-like computations and human-like learning and intelligence. We believe that for an artificial model to be a viable model of learning and intelligence, the model must be capable of matching the experimental results of its natural counterpart. Loosely stated, the artificial model must be able to mimic and exhibit the behavior of the natural one. Artificial neural networks have problems that limit their ability to mimic the behavior of the natural ones; one such problem is the topic of this paper. In the paper we are limiting our focus to the back-propagation (BP) model [8].Artificial neural networks have a similar topography to directed graphs (see Figure 1). The nodes in the graph are known as units or neurons. The edges of the graph are known as connections or synapses. The units are linked to each other via connections. The connections are unidirectional. Each unit has output and input values. Each connection has a weight value. The weights of the connections represent the long term memory (LTM) of the network. LTM contains the learning the network has accomplished. LTM is the perma...

show abstract

Section: Discussionsupporting

confidence: 70%

Section: Analysis Of the Weightsmentioning

confidence: 91%

Section: Analysis Of the Weightsmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

<title>Negative transfer problem in neural networks</title>

Abunawass

1992

SPIE Proceedings

View full text Add to dashboard Cite

show abstract

The negative transfer problem in neural networks: a solution

Abunawass

1991

[Proceedings] 1991 IEEE International Joint Conference on Neural Networks

View full text Add to dashboard Cite

Biologically based machine learning paradigms

Abunawass

1992

Proceedings of the Twenty-Third SIGCSE Technical Symposium on Computer Science Education

View full text Add to dashboard Cite

This paper describes an introductory course on biologically based sub-symbolic machine learning paradigms. Specifically, this paper covers Artificial Neural Networks, Genetic Algorithms and Genetics-Based Machine Learning. It provides the structure, motivation, content, texts and tools for the course. This course is suitable for an upper division undergraduate level course or as an introductory graduate course. The paper includes a section on bibliographical references to aid the instructor in preparing for this course.

show abstract

Cumulative negative transfer during successive training: analysis of a second sequential learning problem

Cited by 4 publications

References 0 publications

<title>Negative transfer problem in neural networks</title>

<title>Negative transfer problem in neural networks</title>

The negative transfer problem in neural networks: a solution

Biologically based machine learning paradigms

Contact Info

Product

Resources

About