Compressive Regularized Discriminant Analysis of High-Dimensional Data with Applications to Microarray Studies

Abstract

We propose a modification of linear discriminant analysis, referred to as compressive regularized discriminant analysis (CRDA), for analysis of high-dimensional datasets. CRDA is especially designed for feature elimination purpose and can be used as gene selection method in microarray studies. CRDA lends ideas from ℓ q,1 norm minimization algorithms in the multiple measurement vectors (MMV) model and utilizes joint-sparsity promoting hard thresholding for feature elimination. A regularization of the sample covariance matrix is also needed as we consider the challenging scenario where the number of features (variables) is comparable or exceeding the sample size of the training dataset. A simulation study and four examples of real life microarray datasets evaluate the performances of CRDA based classifiers. Overall, the proposed method gives fewer misclassification errors than its competitors, while at the same time achieving accurate feature selection.

Optimal Shrinkage Covariance Matrix Estimation Under Random Sampling From Elliptical Distributions

“…Consequently, an estimator of β Gau o is obtained by substituting an estimatorγ in place of γ in (23). Recall that an estimator of α Gau o is then obtained asα Gau o = (1 −β Gau o )tr(S)/p.…”

A Feature Optimization Approach Based on Inter-Class and Intra-Class Distance for Ship Type Classification

“…Fbank and MFCC are not only common features in traditional target classification, but also widely used in classification tasks based on deep learning. Generally, traditional feature optimization methods such as cepstral mean and variance normalization (CMVN) or L1/L2 normalization [ 13 , 14 ] can improve feature discrimination, but they just refine the features at the mathematical level and ignore the degree of association within the class.…”

A Compressive Classification Framework for High-Dimensional Data

“…Shrunken Centroids Regularized Discriminant Analysis (SCRDA) [3] uses an estimator Σ = αS + (1 − α)I and element-wise soft-shrinkage. SCRDA can be expressed in the form (10) with…”

“…We split the training data set X into Q folds and use Q − 1 folds as a training set and one fold as the validation set. We fit the model (CRDA classifier as defined by (10) and ( 11)) on the training set using ϕ ∈ [ϕ] and K ∈ [K] and compute the misclassification error rate on the validation set. This procedure is repeated so that each of the Q folds is left out in turn as a validation set, and misclassification rates are accumulated.…”

“…The results illustrate that overall, the proposed CRDA method gives fewer misclassification errors than its competitors while achieving, at the same time, accurate feature selection with minimal computational efforts. Preliminary results of the CRDA approach has appeared in [10].…”