Automatic Induction of Projection Pursuit Indices

Abstract

Projection techniques are frequently used as the principal means for the implementation of feature extraction and dimensionality reduction for machine learning applications. A well established and broad class of such projection techniques is the projection pursuit (PP). Its core design parameter is a projection index, which is the driving force in obtaining the transformation function via optimization, and represents in an explicit or implicit way the user's perception of the useful information contained within the datasets. This paper seeks to address the problem related to the design of PP index functions for the linear feature extraction case. We achieve this using an evolutionary search framework, capable of building new indices to fit the properties of the available datasets. The high expressive power of this framework is sustained by a rich set of function primitives. The performance of several PP indices previously proposed by human experts is compared with these automatically generated indices for the task of classification, and results show a decrease in the classification errors.

A Projection Pursuit framework for supervised dimension reduction of high dimensional small sample datasets

“…However, it has been observed [41,38] that the successive application of this procedure (as done in the original SPP) may lead to data distortions, implying that an optimum found in residual data may not be longer related to any relevant aspect of the original data. Recently, Zhang and Chan [41,28] proposed an alternative approach to structure removal, which uses the orthogonal complement space concept. 1 In those works, the residual data is obtained projecting the current data onto the orthogonal complement of the found projection vector, which avoid data distortions and also ensures orthogonality of the projection bases.…”

“…The original structure removal procedure of SPP is replaced by these procedures, which are based on the work of RodriguezMartinez et al [28]. Deflation prepares the search space for the next iteration and is executed after PPGA completion (except when the required number of bases is attained, in which case the loop is halted).…”

“…A key advantage of PP is its flexibility to fit different pattern recognition tasks, depending on the PP index used. For example, PP can be used to perform clustering analysis [19,20], classification [21][22][23][24], regression analysis [25] and density estimation [26] (some reviews of PP indexes can be found in [21,27,28]). Another advantage of PP is its out-of-sample mapping capability, that is, the possibility to map new examples in the projection space after the construction of it.…”

Reduction of Dimension and Size of Data Set by Parallel Fast Simulated Annealing

“…Here are often used statistical methods [6] as well as heuristic procedures of optimization, e.g. evolutionary algorithms [7]. A reduction in data set size can be realized with a wide range of sampling or grouping methods.…”

Hybrid Hierarchical Clustering — Piecewise Aggregate Approximation, with Applications

“…12 PP seeks to identify projections that optimize a de¯ned projection index that represents, in an explicit or implicit form, useful information contained within data series. 13 It is useful particularly for data sets that are non-Gaussian, but is much more computationally intensive than PCA.…”