Estimation of the $l_2$-norm and testing in sparse linear regression with unknown variance

Carpentier, Alexandra; Collier, Olivier; Comminges, Laëtitia; Tsybakov, Alexandre B.; Wang, Yuhao

doi:10.48550/arxiv.2010.13679

Cited by 1 publication

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…( 16), falls into the range n r. When r 0 and r 1 are of the same order, confident testing in the well-separated regime thus only requires the sample size n = O(min{r 0 , r 1 }). One can then run (Lin) with such sample sizes to begin with, thus effectively reducing it to (23) and avoid the computational burden of performing projections.…”

Section: Discussion Of Implicationsmentioning

confidence: 99%

“…• On the other hand, the projection step is crucial for ill-separated problems, i.e., when ∆ 1/ √ r. Indeed, in this case the sample complexity bound in (16), falls beyond the range n r, and test (23) becomes suboptimal. More precisely, one can easily verify (by mimicking the decomposition in the proof of Proposition 1) that the sample complexity for test (23) is…”

Section: Discussion Of Implicationsmentioning

confidence: 99%

“…• In the general M -estimation setup (as defined in ( 1)-( 3)), we naturally generalize test (23) to…”

Section: Discussion Of Implicationsmentioning

confidence: 99%

“…Still, one important direction is signal testing in linear regression [19,20,21]. Variants of this problem were considered when the signal is sparse and the noise either known [22] or unknown [23]. Other works test, in the same flavor, sparsity of the signal [24] or some component of the signal [25].…”

Section: Overview Of Technical Contributions Literature and Paper Org...mentioning

confidence: 99%

See 3 more Smart Citations

Near-Optimal Procedures for Model Discrimination with Non-Disclosure Properties

Ostrovskii¹,

Ndaoud²,

Javanmard³

et al. 2020

Preprint

View full text Add to dashboard Cite

Let θ 0 , θ 1 ∈ R d be the population risk minimizers associated to some loss : R d × Z → R and two distributions P 0 , P 1 on Z. Our work is motivated by the following question: Given i.i.d. samples from P 0 and P 1 , what sample sizes are sufficient and necessary to distinguish between the two hypotheses θ * = θ 0 and θ * = θ 1 for given θ * ∈ {θ 0 , θ 1 }?Making the first steps towards answering this question in full generality, we first consider the case of a well-specified linear model with squared loss. Here we provide matching upper and lower bounds on the sample complexity, showing it to be min{1/∆ 2 , √ r/∆} up to a constant factor, where ∆ is a measure of separation between P 0 and P 1 , and r is the rank of the design covariance matrix. This bound is dimension-independent, and rank-independent for large enough separation. We then extend this result in two directions: (i) for the general parametric setup in asymptotic regime; (ii) for generalized linear models in the small-sample regime n r and under weak moment assumptions. In both cases, we derive sample complexity bounds of a similar form, even under misspecification. In fact, our testing procedures only access θ * through a certain functional of empirical risk. In addition, the number of observations that allows to reach statistical confidence in our tests does not allow to "resolve" the two models -that is, recover θ 0 , θ 1 up to O(∆) prediction accuracy. These two properties allow to use our framework in applied tasks where one would like to identify a prediction model, which can be proprietary, while guaranteeing that the model cannot be actually inferred by the agent performing identification. * Equal contribution of the first two authors. 1 For this expository discussion, we define the sample complexity of a (binary) testing problem as the size of an i.i.d. sample for which there exists a test with testing errors of both types at most 0.05.

show abstract

Section: Discussion Of Implicationsmentioning

confidence: 99%