Evolving Linear Discriminant in a Continuously Growing Dimensional Space for Incremental Attribute Learning

Abstract

Abstract. Feature Ordering is a unique preprocessing step in Incremental Attribute Learning (IAL), where features are gradually trained one after another. In previous studies, feature ordering derived based upon each individual feature's contribution is time-consuming. This study attempts to develop an efficient feature ordering algorithm by some evolutionary approaches. The feature ordering algorithm presented in this paper is based on a criterion of maximum mean of feature discriminability. Experimental results derived by ITID, a neural IAL algorithm, show that such a feature ordering algorithm has a higher probability to obtain the lowest classification error rate with datasets from UCI Machine Learning Repository.

Neural Incremental Attribute Learning in Groups

“…Previous studies discovered that feature ordering relies on feature's discriminative ability. There are two approaches of feature ordering: contribution-based feature ranking 6,11,12 and metricbased feature ranking [21][22][23][24][25][26] . Both approaches can exhibit better performance than the conventional batch-training approach.…”

“…In previous research, several metrics for feature ordering have been discovered, such as mRMR 25,27 , Entropy 23 , Single Discriminability (SD) 21 , Evolving Linear Discriminant 24 and Fisher Score 26 . Furthermore, based on the accumulative feature discrimination ability, the Evolving Linear Discriminant exhibits the most stable performance.…”

“…Maximum Mean Discriminative Criterion (MMDC) 24 is very useful to judge whether an IAL feature ordering is optimum or not. According to this criterion, an optimum feature ordering should have the greatest feature discrimination ability all the time when features are successively imported into IAL training system.…”

Output Effect Evaluation Based on Input Features in Neural Incremental Attribute Learning for Better Classification Performance

“…It can be used as a predictive tool to evaluate the final classification performance. There are many feature discrimination ability estimation approaches for feature ordering [14][15][16][17][18]22]. Usually, feature discrimination ability can be derived based on each single feature's contribution or some statistical metrics.…”

“…For example, Incremental Learning in terms of Input Attributes (ILIA) [9] and Incremental Training with an Increasing input Dimension (ITID) [10] have been shown to be applicable for achieving better performance by neural network based IAL. The other factor is feature ordering, a unique preprocessing in IAL [14][15][16][17][18]. In comparison with the results derived by conventional batch training machine learning approaches, both the structure and the preprocessing of feature ordering in IAL can bring positive efforts on the improvement of classification accuracy.…”