Universality laws for randomized dimension reduction, with applications

Abstract

Dimension reduction is the process of embedding high-dimensional data into a lower dimensional space to facilitate its analysis. In the Euclidean setting, one fundamental technique for dimension reduction is to apply a random linear map to the data. This dimension reduction procedure succeeds when it preserves certain geometric features of the set. The question is how large the embedding dimension must be to ensure that randomized dimension reduction succeeds with high probability.This paper studies a natural family of randomized dimension reduction maps and a large class of data sets. It proves that there is a phase transition in the success probability of the dimension reduction map as the embedding dimension increases. For a given data set, the location of the phase transition is the same for all maps in this family. Furthermore, each map has the same stability properties, as quantified through the restricted minimum singular value. These results can be viewed as new universality laws in high-dimensional stochastic geometry.Universality laws for randomized dimension reduction have many applications in applied mathematics, signal processing, and statistics. They yield design principles for numerical linear algebra algorithms, for compressed sensing measurement ensembles, and for random linear codes. Furthermore, these results have implications for the performance of statistical estimation methods under a large class of random experimental designs.

Survival Ensemble with Sparse Random Projections for Censored Clinical and Gene Expression Data

“…The reason why SERSP is beaten by Cocktail on TCGA might be that random projection approach may not work well when the embedding dimension (p1 or p2 ) is smaller than the so-called statistical dimension [35]. As in the experiments, we simply set the embedding dimension to be the root of dimension of the variables, a fine tuning of parameter p1 or p2 would improve SERSP in this case.…”

Concentration of the Frobenius Norm of Generalized Matrix Inverses

“…Gaussian random matrices may appear as a serious restriction. It is known, however, that Gaussian matrices are a representative of a large class of random matrix models for which many relevant functionals are universal-they concentrate around the same value for a given matrix size [11,23]. 7 Although it is tempting to justify our model choice by universality, in the case of sparse pseudoinverses we must proceed with care.…”

Estimation error of the constrained lasso

“…Proposition III.2, which shows the validity of the relaxed RSC condition, can be easily extended for design matrices whose rows are not necessarily subgaussian, with a possibly worse sample complexity bound compared to (3). The interested reader is referred to [6], [14], [19] for the details.…”