Incremental learning methods with retrieving of interfered patterns

Abstract

There are many cases that a neural-network-based system must memorize some new patterns incrementally. However, if the network learns the new patterns only by referring to them, it probably forgets old memorized patterns, since parameters in the network usually correlate not only to the old memories but also to the new patterns. A certain way to avoid the loss of memories is to learn the new patterns with all memorized patterns. It needs, however, a large computational power. To solve this problem, we propose incremental learning methods with retrieving interfered patterns (ILRI). In these methods, the system employs a modified version of a resource allocating network (RAN) which is one variation of a generalized radial basis function (GRBF). In ILRI, the RAN learns new patterns with a relearning of a few number of retrieved past patterns that are interfered with the incremental learning. In this paper, we construct ILRI in two steps. In the first step, we construct a system which searches the interfered patterns from past input patterns stored in a database. In the second step, we improve the first system in such a way that the system does not need the database. In this case, the system regenerates the input patterns approximately in a random manner. The simulation results show that these two systems have almost the same ability, and the generalization ability is higher than other similar systems using neural networks and k-nearest neighbors.

Incremental Multiple Objective Genetic Algorithms

“…Fu et al [19] proposed an incremental backpropagation learning network that employs bounded weight modification and structural adaptation learning rules and applies initial knowledge to constrain the learning process. Yamauchi et al [4] proposed incremental learning methods for retrieving interfered patterns. In their methods, a neural network learns new patterns with a relearning of a few number of retrieved past patterns that interfere with the new patterns.…”

“…Thus, the final chromosome can be encoded as a string consisting of characters. According to the above encoding mechanism, each chromosome will consist of characters, where (4) If all the antecedent elements in a rule are inactive, this rule will be regarded as a noncontributing rule. With this mechanism, our approach has the feature of variable-length GAs so that the number of active rules in a rule set can be flexible.…”

“…However, most work explores incremental learning with statistical algorithms [3] or neural networks [4], and none touches on the use of evolutionary algorithms. This paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning.…”

“…Pattern classification problems have been widely used as traditional formulation of machine learning problems and researched with different machine learning approaches including statistical methods [3], neural networks [4]- [6], genetic algorithms [7], [8], fuzzy sets [9], [10], and cellular automata [11]. In recent years, hybrid approaches are also emerging, as they are better in solving more complicated classification problems Manuscript received April 5, 2004; revised August 7, 2004 andDecember 6, 2004.…”

Ordered incremental training with genetic algorithms

“…Yamauchi et al 10 proposed incremental learning methods for retrieving interfered patterns. In their methods, a neural network learns new patterns with relearning of a small number of retrieved past patterns that interfere with the new patterns.…”