These notes survey and explore an emerging method, which we call the low-degree method, for predicting and understanding statistical-versus-computational tradeoffs in high-dimensional inference problems. In short, the method posits that a certain quantity -the second moment of the low-degree likelihood ratio -gives insight into how much computational time is required to solve a given hypothesis testing problem, which can in turn be used to predict the computational hardness of a variety of statistical inference tasks. While this method originated in the study of the sum-of-squares (SoS) hierarchy of convex programs, we present a self-contained introduction that does not require knowledge of SoS. In addition to showing how to carry out predictions using the method, we include a discussion investigating both rigorous and conjectural consequences of these predictions.These notes include some new results, simplified proofs, and refined conjectures. For instance, we point out a formal connection between spectral methods and the low-degree likelihood ratio, and we give a sharp low-degree lower bound against subexponential-time algorithms for tensor PCA.

We study the computational cost of recovering a unit-norm sparse principal component x ∈ R n planted in a random matrix, in either the Wigner or Wishart spiked model (observing either W + λxx ⊤ with W drawn from the Gaussian orthogonal ensemble, or N independent samples from N (0, I n + βxx ⊤ ), respectively). Prior work has shown that when the signal-tonoise ratio (λ or β N/n, respectively) is a small constant and the fraction of nonzero entries in the planted vector isWhile it is possible to recover x in exponential time under the weaker condition ρ ≪ 1, it is believed that polynomial-time recovery is impossible unless ρ 1/ √ n. We investigate the precise amount of time required for recovery in the "possible but hard" regime 1/ √ n ≪ ρ ≪ 1 by exploring the power of subexponential-time algorithms, i.e., algorithms running in time exp(n δ ) for some constant δ ∈ (0, 1). For any 1/ √ n ≪ ρ ≪ 1, we give a recovery algorithm with runtime roughly exp(ρ 2 n), demonstrating a smooth tradeoff between sparsity and runtime. Our family of algorithms interpolates smoothly between two existing algorithms: the polynomial-time diagonal thresholding algorithm and the exp(ρn)-time exhaustive search algorithm. Furthermore, by analyzing the low-degree likelihood ratio, we give rigorous evidence suggesting that the tradeoff achieved by our algorithms is optimal.

No abstract

We study when low coordinate degree functions (LCDF)-linear combinations of functions depending on small subsets of entries of a vector-can hypothesis test between high-dimensional probability measures. These functions are a generalization, proposed in Hopkins' 2018 thesis but seldom studied since, of low degree polynomials (LDP), a class widely used in recent literature as a proxy for all efficient algorithms for tasks in statistics and optimization. Instead of the orthogonal polynomial decompositions used in LDP calculations, our analysis of LCDF is based on the Efron-Stein or ANOVA decomposition, making it much more broadly applicable. By way of illustration, we prove channel universality for the success of LCDF in testing for the presence of sufficiently "dilute" random signals through noisy channels: the efficacy of LCDF depends on the channel only through the scalar Fisher information for a class of channels including nearly arbitrary additive i.i.d. noise and nearly arbitrary exponential families. As applications, we extend lower bounds against LDP for spiked matrix and tensor models under additive Gaussian noise to lower bounds against LCDF under general noisy channels. We also give a simple and unified treatment of the effect of censoring models by erasing observations at random and of quantizing models by taking the sign of the observations. These results are the first computational lower bounds against any large class of algorithms for all of these models when the channel is not one of a few special cases, and thereby give the first substantial evidence for the universality of several statistical-to-computational gaps.