Minimax Lower Bounds for Noisy Matrix Completion Under Sparse Factor Models

Abstract

This paper examines fundamental error characteristics for a general class of matrix completion problems, where the matrix of interest is a product of two a priori unknown matrices, one of which is sparse, and the observations are noisy. Our main contributions come in the form of minimax lower bounds for the expected per-element squared error for this problem under several common noise models. Specifically, we analyze scenarios where the corruptions are characterized by additive Gaussian noise or additive heavier-tailed (Laplace) noise, Poisson-distributed observations, and highly-quantized (e.g., one-bit) observations, as instances of our general result. Our results establish that the error bounds derived in (Soni et al., 2016) for complexity-regularized maximum likelihood estimators achieve, up to multiplicative constants and logarithmic factors, the minimax error rates in each of these noise scenarios, provided that the nominal number of observations is large enough, and the sparse factor has (on an average) at least one non-zero per column. Index TermsMatrix completion, dictionary learning, minimax lower bounds I. INTRODUCTIONThe matrix completion problem involves imputing the missing values of a matrix from an incomplete, and possibly noisy sampling of its entries. In general, without making any assumption about the entries of the matrix, the matrix completion problem is ill-posed and it is impossible to recover the matrix uniquely. However, if the matrix to be recovered has some intrinsic structure (e.g., low rank structure), it is possible to design algorithms that exactly estimate the missing entries. Indeed, the performance low-rank matrix completion methods have been extensively studied in noiseless settings [1]-[5], in noisy settings where the observations are affected by additive noise [6]-[12], and in settings where the observations are non-linear (e.g., highly-quantized or Poisson distributed observation) functions of the underlying matrix entry (see, [13]-[15]). Recent works which explore robust recovery of low-rank matrices under malicious sparse corruptions include [16]-[19].A notable advantage of using low-rank models is that the estimation strategies involved in completing such matrices can be cast into efficient convex methods which are well-understood and suitable to analyses. The

Sparse Nonnegative Tensor Factorization and Completion with Noisy Observations

“…Now the lower bound of candidate estimators for Poisson observations is given in the following proposition, whose proof follows a similar line of the matrix case in [42,Theorem 6]. For the sake of completeness, we give it here.…”

“…where the first inequality follows from (27), the second inequality follows from (43), and the last inequality follows from (42). Note that…”

“…They showed that the error bounds of estimators of sparse NMF are lower than those of NMF [48]. Furthermore, Sambasivan et al [42] derived the minimax lower bounds of the expected per-element square error under general noise observations. Due to exploiting the intrinsic structure of the underlying tensor data, which contains correlation in different modes, NTF has also been widely applied in a variety of fields, see, e.g., [5,16,34,39,50].…”

Sparse Nonnegative Tensor Factorization and Completion With Noisy Observations

“…This demonstrates the upper bound in Theorem 4.1 is nearly optimal. [26,43], see also [54]. The key issue of the proof is to construct the packing sets for the core tensors and sparse factor matrices, respectively.…”

“…In theory, Soni et al [46] proposed a noisy matrix completion method under a class of general noise models, and established the error bound of the estimator of their proposed model, which can reduce to sparse NMF and completion under nonnegative constraints. Moreover, Sambasivan et al [43] showed the error bound in [46] achieves minimax error rates up to multiplicative constants and logarithmic factors. However, for multi-dimensional data, the matrix based method may destroy the structure of a tensor via unfolding a tensor into a matrix.…”