Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees

Abstract

In this paper, we investigate the matrix estimation problem in multi-response regression with measurement errors. A nonconvex error-corrected estimator is proposed to estimate the matrix parameter via a combination of the loss function and the nuclear norm regularization. Then under the low-rank constraint, we analyse the statistical and computational theoretical properties of global solution of the nonconvex regularized estimator from a general point. In the statistical aspect, we establish the recovery bound for the global solution of the nonconvex estimator, under restricted strong convexity on the loss function. In the computational aspect, we solve the nonconvex optimization problem via the proximal gradient method. The algorithm is proved to converge to a near-global solution and achieve a linear convergence rate. In addition, we also establish sufficient conditions for the general results to be held for specific types of corruptions, including the additive noise and missing data. Probabilistic consequences are obtained by applying the general results. Finally, we demonstrate our theoretical consequences by several numerical experiments on the corrupted errors-in-variables multi-response regression models. Simulation results show remarkable consistency with our theory under high-dimensional scaling.

Minimax Lower Bounds for High-Dimensional Multi-Response Errors-in-Variables Regression

“…Then we establish lower bounds on the minimax loss function in terms of the squared Frobenius norm for a certain class of lowrank matrices. This lower bound agrees with the upper bound given before in our another work [25] up to constant factors, implying the optimality of the proposed estimator therein. Moreover, the minimax lower bound recaptures the rate provided under the clean covariate assumption in previous literatures [12,14,15], a result that further indicates that though in the more realistic errors-in-variables situation, no more samples are required so as to achieve a rate-optimal estimation.…”

“…Theorem 1 tells us that in the additive noise case, with high probability, about max{d 1 , d 2 }R 0 number of samples are required to estimate a d 1 × d 2 matrix of rank R 0 consistently by any method. Note that the lower bound(10) agrees with the upper bound obtained in our another work[25, Theorem 1] when λ = Ω…”

“…In this section, we provide the lower bound on the minimax risk which holds with high probability. This lower bound implies that the achievable upper bound established in [25, Theorem 1] is sharp and thus provide the information-theoretic foundation that no estimator can perform better than the nonconvex estimator proposed in [25] in the sense of statistical convergence rates.…”

“…The proof is complete. up to constant factors, implying that the proposed error-corrected estimator in [25] is minimax optimal in the additive noise case.…”