RAPTT: An Exact Two-Sample Test in High Dimensions Using Random Projections

Abstract

In 1 high dimensions, the classical Hotelling's T 2 test tends to have low power or becomes undefined due to singularity of the sample covariance matrix. In this paper, this problem is overcome by projecting the data matrix onto lower dimensional subspaces through multiplication by random matrices. We propose RAPTT (RAndom Projection T-Test), an exact test for equality of means of two normal populations based on projected lower dimensional data.RAPTT does not require any constraints on the dimension of the data or the sample size. A simulation study indicates that in high dimensions the power of this test is often greater than that of competing tests. The advantage of RAPTT is illustrated on high-dimensional gene expression data involving the discrimination of tumor and normal colon tissues.

A Powerful Bayesian Test for Equality of Means in High Dimensions

“…One possible way to achieve this is to sample the entries of R from a distribution with mean zero and variance one. Since our test statistics involves the inversion of R T SR , which is positive definite if R T R = I m (see Lemma 1 of Srivastava et al , 2016), we further restrict our choices to the family of semi-orthogonal matrices. We consider two constructions of the projection matrix.…”

“…RP has entered the frequentist hypothesis testing literature where the T 2 statistics are based on the projected version of the data in ‘large-p-small-n’ setting. See, for example, Lopes et al (2011) and Srivastava et al (2016).…”

An adaptive two-sample test for high-dimensional means

“…As shown theoretically in Fan (1996) , as the dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$p$\end{document} increases, even for simple one-sample testing on the mean of a normal distribution with a known covariance matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$\sigma ^2 I$\end{document} , the standard Wald, score or likelihood ratio tests may have power that decreases to the Type I error rate as the departure from the null hypothesis increases. Several two-sample tests for high-dimensional data have been proposed ( Bai & Saranadasa, 1996 ; Srivastava & Du, 2008 ; Chen & Qin, 2010 ; Cai et al, 2014 ; Gregory et al, 2015 ; Srivastava et al, 2015 ). There are two common types of testing approach when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$p>n$\end{document} : one based on the sum-of-squares of the sample mean differences and the other based on the maximum componentwise sample mean difference.…”

Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states?

“…Step 4 rejects based on an average of p-values. This approach is also taken for tests based on random projections [Srivastava (2014)]. Under the null, the p b should have a uniform distribution.…”