Detecting the direction of a signal on high-dimensional spheres: non-null and Le Cam optimality results

Paindaveine, Davy; Verdebout, Thomas

doi:10.1007/s00440-019-00937-x

Cited by 8 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With perfectly matched data in the spherical domain S p−1 , estimation of an orthogonal matrix W ∈ SO(p) = {A ∈ R p×p : AA T = I p } that transforms the predictors to responses has been referred to as the spherical regression (Chang 1986, 1989, Goodall 1991, Kim 1998, Rosenthal et al 2014, Di Marzio et al 2018. Statistical inference beyond the classical setup of fixed dimension p has also been considered recently (Paindaveine & Verdebout 2017). However, the current literature is based on the assumption that the response and predictor are correctly linked.…”

Section: Spherical Regression With Mismatched Datamentioning

confidence: 99%

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Shi

Cai

2020

Journal of the American Statistical Association

View full text Add to dashboard Cite

Motivated by a series of applications in data integration, language translation, bioinformatics, and computer vision, we consider spherical regression with two sets of unit-length vectors when the data are corrupted by a small fraction of mismatch in the response-predictor pairs. We propose a three-step algorithm in which we initialize the parameters by solving an orthogonal Procrustes problem to estimate a translation matrix W ignoring the mismatch. We then estimate a mapping matrix aiming to correct the mismatch using hard-thresholding to induce sparsity, while incorporating potential group information. We eventually obtain a refined estimate for W by removing the estimated mismatched pairs. We derive the error bound for the initial estimate of W in both fixed and high-dimensional setting. We demonstrate that the refined estimate of W achieves an error rate that is as good as if no mismatch is present. We show that our mapping recovery method not only correctly distinguishes one-to-one and one-to-many correspondences, but also consistently identifies the matched pairs and estimates the weight vector for combined correspondence. We examine the finite sample performance of the proposed method via extensive simulation studies, and with application to the unsupervised translation of medical codes using electronic health records data.

show abstract

Section: Spherical Regression With Mismatched Datamentioning

confidence: 99%

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Shi

Cai

2020

Journal of the American Statistical Association

View full text Add to dashboard Cite

show abstract

“…Rotational symmetry has also been extensively adopted in the literature as a core assumption for performing inference with directional data. A (far from exhaustive) list of references illustrating this follows: Rivest (1989), Ko and Chang (1993), and Chang and Rivest (2001) considered regression and M-estimation under rotationally symmetric assumptions; Lo and Cabrera (1987) considered Bayes procedures for rotationally symmetric models; Larsen et al (2002) considered von Mises-Fisher likelihood ratios; Tsai and Sen (2007), , and Paindaveine and Verdebout (2015) proposed rank tests and estimators for the mode of a rotationally symmetric distribution; proposed a concept of quantiles for rotationally symmetric distributions; Paindaveine and Verdebout (2017) considered inference for the direction of weak rotationally symmetric signals, whereas Paindaveine and Verdebout (2019) tackled high-dimensional hypothesis testing in the same framework.…”

Section: Motivationmentioning

confidence: 99%

On Optimal Tests for Rotational Symmetry Against New Classes of Hyperspherical Distributions

García‐Portugués

Paindaveine

Verdebout

2019

Journal of the American Statistical Association

Self Cite

View full text Add to dashboard Cite

Motivated by the central role played by rotationally symmetric distributions in directional statistics, we consider the problem of testing rotational symmetry on the hypersphere. We adopt a semiparametric approach and tackle problems where the location of the symmetry axis is either specified or unspecified. For each problem, we define two tests and study their asymptotic properties under very mild conditions. We introduce two new classes of directional distributions that extend the rotationally symmetric class and are of independent interest. We prove that each test is locally asymptotically maximin, in the Le Cam sense, for one kind of the alternatives given by the new classes of distributions, both for specified and unspecified symmetry axis. The tests, aimed to detect location-like and scatter-like alternatives, are combined into convenient hybrid tests that are consistent against both alternatives. We perform Monte Carlo experiments that illustrate the finite-sample performances of the proposed tests and their agreement with the asymptotic results. Finally, the practical relevance of our tests is illustrated on a real data application from astronomy. The R package rotasym implements the proposed tests and allows practitioners to reproduce the data application.

show abstract

“…In line with this, inference on high-dimensional spheres has been much considered in the last decades. While other inference problems have also been considered (see, e.g., [38] for high-dimensional location problems), most of the focus has been on the problem of testing uniformity on the unit sphere. An asymptotic investigation of this problem requires that the dimension p n diverges to infinity with n (see, e.g., [9] and [15]), which makes it necessary to adopt the triangular framework above (some other works, including [11,12], actually rather consider a fixed-n large-p asymptotic scenario; see also the monograph [13]).…”

Section: Introductionmentioning

confidence: 99%

Testing uniformity on high-dimensional spheres: The non-null behaviour of the Bingham test

Cutting¹,

Paindaveine

Verdebout

2022

Ann. Inst. H. Poincaré Probab. Statist.

Self Cite

View full text Add to dashboard Cite

Testing uniformity on the unit sphere of R p is a fundamental problem in directional statistics. In the framework of axial data, the most classical test of uniformity is the Bingham [8] test. Remarkably, this test does not need any modification to meet asymptotically the target null size in high-dimensional scenarios where p = pn diverges to infinity with the sample size n. However, while the non-null asymptotic behaviour of the Bingham test is well understood in standard asymptotic scenarios where n diverges to infinity with p fixed, nothing is known on the power of this test in high dimensions, not even under standard parametric alternatives such as Watson distributions. In this work, we therefore study the non-null behaviour of the Bingham test in high dimensions. First, we consider a semiparametric class of alternatives that includes Watson alternatives and we derive a local asymptotic normality (LAN) property. An application of Le Cam's third lemma reveals that the Bingham test is blind to the corresponding contiguous alternatives, though. By using martingale central limit theorems, we therefore study the non-null behaviour of the Bingham test under more severe alternatives. Far from restricting to the aforementioned semiparametric alternatives, our results cover a broad class of rotationally symmetric alternatives, which allows us to consider non-axial alternatives, too. In every distributional framework we consider, the "detection threshold" of the Bingham test is identified and a comparison with the classical test of uniformity for non-axial data, namely the Rayleigh [40] test, is made possible. In the framework of axial data, we derive a lower bound on the minimax separation rate and establish that the Bingham test is minimax rate-optimal in the class of Watson distributions.

show abstract

Detecting the direction of a signal on high-dimensional spheres: non-null and Le Cam optimality results

Cited by 8 publications

References 33 publications

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

On Optimal Tests for Rotational Symmetry Against New Classes of Hyperspherical Distributions

Testing uniformity on high-dimensional spheres: The non-null behaviour of the Bingham test

Contact Info

Product

Resources

About