Pareto models for discriminative multiclass linear dimensionality reduction

Abstract

Linear Discriminant Analysis (LDA) is a popular tool for multiclass discriminative dimensionality reduction. However, LDA suffers from two major drawbacks:(i) it only optimizes the Bayes error for two-class problems where each class is a unimodal Gaussian with different mean, but both classes have equal full rank covariance matrices, and (ii) the multiclass extension does not maximize each pairwise distance between the classes, but rather maximizes the sum of all these pairwise distances. This typically results in serious overlapping in the projected space between classes that are close to each other in the input space, and degrades classification performance.To solve these two problems, this paper proposes Pareto Discriminant Analysis (PARDA). First, PARDA explicitly models each class as a multidimensional Gaussian distribution (or as a mixture of Gaussians) with a full rank covariance matrix. Second, PARDA decomposes the multiclass problem to a set of pairwise objective functions representing the pairwise distance between different classes.

Testing the accuracy of species distribution models based on community science data

“…Because my study sites are under-surveyed areas, I have limited absence data and rely entirely on presences. Another advantage is that using this method makes it easy to find outliers as they are at the edges of the ellipsoid (Abou-Moustafa & Ferrie, 2007; Sun & Freund, 2004). To remove the outliers and to create binary prediction models (i.e., SDMs), I used thresholds of 100%, 90%, and 75% data inclusion.…”

Application of MEEMD in post‐processing of dimensionality reduction methods for face recognition

“…Xu et al [9] proposed a novel transfer subspace learning method (the subspace learning problem is assigned as a sparse and low-rank minimisation problem). Abou-Moustafa et al [10] proposed a Pareto discriminant analysis, to maximise each pairwise distance by decomposing the multiclass problem into a set of objective functions.…”

“…Abou‐Moustafa et al . [10] proposed a Pareto discriminant analysis, to maximise each pairwise distance by decomposing the multiclass problem into a set of objective functions.…”

Regularized coplanar discriminant analysis for dimensionality reduction

“…Oh et al presented a generalized mean PCA [11], overcoming the problem that PCA is prone to outliers included in the training set. Abou-Moustafa et al presented a Pareto LDA [12], to maximize each pairwise distance to maximally separate all class mean. Ghassabeh et al proposed a fast incremental LDA [13], which accelerates the convergence rate of the incremental LDA algorithm.…”