Dimension Reduction for Hypothesis Testing in Worst-Case Scenarios

Abstract

This paper considers a "one among many" detection problem, where one has to discriminate between pure noise and one among alternatives that are known up to an amplitude factor. Two issues linked to high dimensionality arise in this framework. First, the computational complexity associated to the Generalized Likelihood Ratio (GLR) with the constraint of sparsity-one inflates linearly with , which can be an obstacle when multiple data sets have to be tested. Second, standard procedures based on dictionary learning aimed at reducing the dimensionality may suffer from severe power losses for some alternatives, thus suggesting a worst-case scenario strategy. In the case where the learned dictionary has column, we show that the exact solution of the resulting detection problem, which can be formulated as a minimax problem, can be obtained by Quadratic Programming. Because it allows a better sampling of the diversity of the alternatives, the case is expected to improve the detection performances over the case . The worst-case analysis of this case, which is more involved, leads us to propose two "minimax learning algorithms". Numerical results show that these algorithms indeed allow to increase performances over the case and are in fact comparable to the GLR using the full set of alternatives, while being computationally simpler.

ORIGIN: Blind detection of faint emission line galaxies in MUSE datacubes

“…These signatures essentially take the form of '3D blobs' corresponding to the convolution of a spectral line profile by the spatial PSF of the instrument. Several matched filtering (Herenz et al 2017) or Generalized Likelihood Ratio (GLR) approaches (Paris et al 2013;Suleiman et al 2013Suleiman et al , 2014 have been devised for that purpose. Such approaches lead to filter the data cube with a library of possible signatures, built as spectral line profiles spread spatially by the PSF, and to normalize the result.…”

Convergence rates for smooth k-means change-point detection

“…In a wide range of areas, change-point problems may occur in a high-dimensional context. This is the case for instance in the analysis of network traffic data Levy-Leduc and Roueff [2009], Lung-Yut-Fong et al [2012], in bioinformatics, when studying copy-number variation Bleakley and Vert [2011], Zhang et al [2010], in astrostatistics Bourguignon et al [2011], Suleiman et al [2014], Meillier et al [2016] or in multimedia indexation Harchaoui et al [2009]. In these practical applications, the number of observations is relatively small compared with their dimension, with the change-point possibly occurring only for a few components.…”

Introduction to optimization with applications in astronomy and astrophysics

“…In Astronomy, this problem is for instance encountered in minimax detection of spectral profiles [Suleiman et al, 2014]. In this application, we are given a library of spectral profiles {h i ∈ R n , i = 1, .…”