Bayes-optimality motivated linear and multilayered perceptron-based dimensionality reduction

Abstract

Dimensionality reduction is the process of mapping high-dimension patterns to a lower dimension subspace. When done prior to classification, estimates obtained in the lower dimension subspace are more reliable. For some classifiers, there is also an improvement in performance due to the removal of the diluting effect of redundant information. A majority of the present approaches to dimensionality reduction are based on scatter matrices or other statistics of the data which do not directly correlate to classification accuracy. The optimality criteria of choice for the purposes of classification is the Bayes error. Usually however, Bayes error is difficult to express analytically. We propose an optimality criteria based on an approximation of the Bayes error and use it to formulate a linear and a nonlinear method of dimensionality reduction. The nonlinear method we propose, relies on using a multilayered perceptron which produces as output the lower dimensional representation. It thus differs from autoassociative like multilayered perceptrons which have been proposed and used for dimensionality reduction. Our results show that the nonlinear method is, as anticipated, superior to the linear method in that it can perform unfolding of a nonlinear manifold. In addition, the nonlinear method we propose provides substantially better lower dimension representation (for classification purposes) than Fisher's linear discriminant (FLD) and two other nonlinear methods of dimensionality reduction that are often used.

Design of pattern recognition system for static security assessment and classification

“…Therefore, a need arises to determine relatively small number of variables, distinctive for each class [28,29]. FS addresses the dimensionality reduction problem, where high dimension patterns are mapped to a lower dimension sub-space [18,19]. FS determines a subset of optimal features to build a good classification model.…”

A New Method for Dimensionality Reduction based on Multivariate Feature Fusion

“…Hereby, there are total nine graphical features of three groups for each class . [7]. Classification accuracy was obtained using a k-NN technique with the optimal value of k selected using cross-validation.…”

A Novel Pattern Classification Method for Multivariate EMG Signals Using Neural Network

“…For instance, optimal criterion of PCA is not directly related to training criteria of its counterparts for pattern classification. Since training of the classification part always aims to realize low error probabilities, it may not always be possible for PCA to extract features in a reduced form, containing high discriminant information [9], [10]. On the other hand, it should be noticed that, in the existing methods, training processes for PCA and the classification part are made separately.…”

“…RD-LLGMN uses orthogonal transformation to project the original feature space into a lowerdimensional space, and then calculates posterior probabilities with a Gaussian mixture model (GMM) in the projected lower-dimensional space for classification. Also, parameters in the network is trained with a single criterion, i.e., minimizing an error probability, it is expected that such training algorithm may yield better classification performance [1], [9].…”