Untitled

Abstract

Neural networks are generally exposed to a dynamic environment where the training patterns or the input attributes (features) will likely be introduced into the current domain incrementally. This paper considers the situation where a new set of input attributes must be considered and added into the existing neural network. The conventional method is to discard the existing network and redesign one from scratch. This approach wastes the old knowledge and the previous effort. In order to reduce computational time, improve generalization accuracy, and enhance intelligence of the learned models, we present ILIA

Evolving Linear Discriminant in a Continuously Growing Dimensional Space for Incremental Attribute Learning

“…Diabetes, Cancer, and Glass have 200 generations in each confirmation round, while Thyroid has 5000 epochs, because of its large feature number. Results based on ITID and feature orderings derived by AD were compared with those results obtained in previous studies, where feature orderings were derived based on original orderings [2], wrappers [4], correlation-based mRMRs [9,10], and conventional approaches [4]. Here, ITID was randomly initialized by 20 different structures, and the final results were the statistical average of these 20 different initial neural networks.…”

“…Particularly, IAL can bring along more accurate results than conventional approaches where features are imported to training by batch. For example, based on UCI datasets, classification errors of Diabetes, Thyroid and Glass derived by ILIA [4] and ITID [1], two neural IAL algorithms, reduced by 8.2%, 14.6% and 12.6%, respectively [1,2]; moreover, based on OIGA, testing error rates derived by IGA of Yeast, Glass and Wine declined by 25.9%, 19.4% and 10.8% [5] in classification. Furthermore, i + Learning and i + LRA, two kinds of IAL decision trees, were employed to run 16 different UCI datasets.…”

“…, a representative of neural IAL based on ILIA [4], is different from traditional approaches which train features by batch. It divides all input dimensions into several sub-dimensions, each of which corresponds to an input feature.…”

Input Space Partitioning for Neural Network Learning

“…The first part means that the current network should be trained for at least a certain number of epochs before a new hidden unit is installed because the error curves will be turbulent in the beginning. The second part means that the current network has been overfit or training has little progress (Guan & Li, 2001).…”

Ordered Incremental Multi-Objective Problem Solving Based on Genetic Algorithms

“…Performance of the obtained incremental algorithms has been shown better than the original ones (Chen, 2003). In fact, the concept of incremental evolution has been also successfully applied in supervised learning, which works both in the input and output spaces (Guan & Li, 2001;Guan & Li, 2002;Guan & Li, 2004;Guan & Liu, 2002;Guan & Liu, 2004;Guan & Zhu, 2003). However, the objective ordering problem still remains.…”