Multivariate Rank-Based Distribution-Free Nonparametric Testing Using Measure Transportation

Deb, Nabarun; Sen, Bodhisattva

doi:10.1080/01621459.2021.1923508

Cited by 65 publications

(32 citation statements)

References 102 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…} all weakly converge to a same probability measure that does not depend on the particular choice of j and k (see Shi et al (2020, Proposition 2.2) as well as Deb and Sen (2021) for a similar setup in the recent independence testing literature). We however do not pursue these tracks but rather leave them to the readers of interest to verify.…”

Section: Theorymentioning

confidence: 97%

Fisher-Pitman permutation tests based on nonparametric Poisson mixtures with application to single cell genomics

Miao¹,

Kong²,

Vinayak³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper investigates the theoretical and empirical performance of Fisher-Pitman-type permutation tests for assessing the equality of unknown Poisson mixture distributions. Building on nonparametric maximum likelihood estimators (NPMLEs) of the mixing distribution, these tests are theoretically shown to be able to adapt to complicated unspecified structures of count data and also consistent against their corresponding ANOVA-type alternatives; the latter is a result in parallel to classic claims made by Robinson (Robinson, 1973). The studied methods are then applied to a single-cell RNA-seq data obtained from different cell types from brain samples of autism subjects and healthy controls; empirically, they unveil genes that are differentially expressed between autism and control subjects yet are missed using common tests. For justifying their use, rate optimality of NPMLEs is also established in settings similar to nonparametric Gaussian (Wu and Yang, 2020a) and binomial mixtures (Tian et al., 2017;Vinayak et al., 2019).

show abstract

Section: Theorymentioning

confidence: 97%

Fisher-Pitman permutation tests based on nonparametric Poisson mixtures with application to single cell genomics

Miao¹,

Kong²,

Vinayak³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Many recent works have focused on obtaining consistent estimators of T 0 using the plug-in principle, see [29,61] (in the semi-discrete problem) and [75,144,44] (in the discrete-discrete problem). In [61], the authors have studied the rate of convergence of the semi-discrete optimal transport map from ν (absolutely continuous) to µ m .…”

Section: Related Workmentioning

confidence: 99%

“…Given two random variables X ∼ µ and Y ∼ ν, where µ, ν are probability measures on R d , d ≥ 1, the problem of finding a "nice" map T 0 (•) such that T 0 (X) ∼ ν has numerous applications in machine learning such as domain adaptation and data integration [67,54,38,37,41,122], dimension reduction [72,13,98], generative models [66,89,96,120], to name a few. Of particular interest is the case when T 0 (•) is obtained by minimizing a cost function, a line of work initiated by Gaspard Monge [106] in 1781 (see (1.1) below), in which case T 0 (•) is termed an optimal transport (OT) map and has applications in shape matching/transfer problems [52,131,32,117], Bayesian statistics [118,51,83,88], econometrics [60,16,31,56,50], nonparametric statistical inference [44,123,124,43,42]; also see [139,140,121] for book-length treatments on the subject. In this paper, we will focus on the OT map obtained using the standard Euclidean cost function, i.e., T 0 := argmin…”

Section: Introductionmentioning

confidence: 99%

Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections

Deb,

Ghosal,

Sen

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Optimal transport maps between two probability distributions µ and ν on R d have found extensive applications in both machine learning and statistics. In practice, these maps need to be estimated from data sampled according to µ and ν. Plug-in estimators are perhaps most popular in estimating transport maps in the field of computational optimal transport. In this paper, we provide a comprehensive analysis of the rates of convergences for general plug-in estimators defined via barycentric projections. Our main contribution is a new stability estimate for barycentric projections which proceeds under minimal smoothness assumptions and can be used to analyze general plug-in estimators. We illustrate the usefulness of this stability estimate by first providing rates of convergence for the natural discrete-discrete and semi-discrete estimators of optimal transport maps. We then use the same stability estimate to show that, under additional smoothness assumptions of Besov type or Sobolev type, wavelet based or kernel smoothed plug-in estimators respectively speed up the rates of convergence and significantly mitigate the curse of dimensionality suffered by the natural discrete-discrete/semi-discrete estimators. As a by-product of our analysis, we also obtain faster rates of convergence for plug-in estimators of W 2 (µ, ν), the Wasserstein distance between µ and ν, under the aforementioned smoothness assumptions, thereby complementing recent results in Chizat et al. (2020). Finally, we illustrate the applicability of our results in obtaining rates of convergence for Wasserstein barycenters between two probability distributions and obtaining asymptotic detection thresholds for some recent optimal-transport based tests of independence.

show abstract

“…To testing the independence between Y ∈ Ê 1 and W ∈ Ê p+q , we will adopt the test proposed in Shi et al (2021a, Equation ( 13)); see Deb and Sen (2021) for a similar result. We will briefly illustrate the idea.…”

Section: Proof Of Theorem 41(ii)mentioning

confidence: 99%

On Azadkia-Chatterjee's conditional dependence coefficient

Shi¹,

Drton²,

Han³

2021

Preprint

View full text Add to dashboard Cite

In recent work, Azadkia and Chatterjee laid out an ingenious approach to defining consistent measures of conditional dependence. Their fully nonparametric approach forms statistics based on ranks and nearest neighbor graphs. The appealing nonparametric consistency of the resulting conditional dependence measure and the associated empirical conditional dependence coefficient has quickly prompted follow-up work that seeks to study its statistical efficiency. In this paper, we take up the framework of conditional randomization tests (CRT) for conditional independence and conduct a power analysis that considers two types of local alternatives, namely, parametric quadratic mean differentiable alternatives and nonparametric Hölder smooth alternatives. Our local power analysis shows that conditional independence tests using the Azadkia-Chatterjee coefficient remain inefficient even when aided with the CRT framework, and serves as motivation to develop variants of the approach; cf. Lin and Han (2021). As a byproduct, we resolve a conjecture of Azadkia and Chatterjee by proving central limit theorems for the considered conditional dependence coefficients, with explicit formulas for the asymptotic variances.

show abstract

Multivariate Rank-Based Distribution-Free Nonparametric Testing Using Measure Transportation

Cited by 65 publications

References 102 publications

Fisher-Pitman permutation tests based on nonparametric Poisson mixtures with application to single cell genomics

Fisher-Pitman permutation tests based on nonparametric Poisson mixtures with application to single cell genomics

Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections

On Azadkia-Chatterjee's conditional dependence coefficient

Contact Info

Product

Resources

About