The Complexity of Distributions

Abstract

Complexity theory typically studies the complexity of computing a function h(x) : {0, 1} m → {0, 1} n of a given input x. A few works have suggested to study the complexity of generating -or sampling -the distribution h(x) for uniform x, given random bits. We further advocate this study, with a new emphasis on lower bounds for restricted computational models. Our main results are:1. Any function f : {0, 1} → {0, 1} n such that (i) each output bit f i depends on o(log n) input bits, and (ii) ≤ log 2 n αn + n 0.99 , has output distribution f (U ) at statistical distance ≥ 1 − 1/n 0.49 from the uniform distribution over n-bit strings of hamming weight αn. We also prove lower bounds for generating (X, b(X)) for boolean b, and in the case in which each bit f i is a small-depth decision tree. These lower bounds seem to be the first of their kind; the proofs use anticoncentration results for the sum of random variables.2. Lower bounds for generating distributions imply succinct data structures lower bounds. As a corollary of (1), we obtain the first lower bound for the membership problem of representing a set S ⊆ [n] of size αn, in the case where 1/α is a power of 2: If queries "i ∈ S?" are answered by non-adaptively probing o(log n) bits, then the representation uses ≥ log 2 n αn + Ω(log n) bits. 3. Upper bounds complementing the bounds in (1) for various settings of parameters. Uniform randomized AC0 circuits of poly(n) size and depth d = O(1) with error can be simulated by uniform randomized AC 0 circuits of poly(n) size and depth d + 1 with error + o(1) using ≤ (log n) O(log log n) random bits. Previous derandomizations [Ajtai and Wigderson '85; Nisan '91] increase the depth by a constant factor, or else have poor seed length.

Fine-Grained Cryptography

“…We note that similar techniques have been used in the past to construct PRGs that fool circuit families of a fixed constant depth -see, for instance, [Vio12].…”

“…In particular, -biased generators with exponentially small give us almost k-wise independent generators for large k, which in turn fool AC 0 circuits by a result of Braverman [Bra10]. This and other techniques have been used in the past to construct PRGs that fool circuits of a fixed constant depth, with the focus generally being on optimising the seed length [Vio12,TX13].…”

Bounded-Depth Circuits Cannot Sample Good Codes

“…Let b be the n-bit Majority function, for n odd. As shown in Viola (2010), there are small AC 0 circuits that generate (x, b(x)) with exponentially small error and using ≥ n log n input random bits. We discuss a possible way to show that small AC 0 circuits cannot generate (x, b(x)) with error 0 (i.e., exactly) and using n random bits, which is open.…”

“…As pointed out in Viola (2010), proving lower bounds approaching 1 on the statistical distance between the output of a circuit and some flat distribution T on {0, 1} n implies data structures lower bounds for storing elements t in the support of T succinctly while retrieving each bit of t efficiently. In particular, one obtains the following lower bound for storing codewords.…”

“…A few works, such as the ones by Jerrum et al (1986), Ambainis et al (2003), Goldreich et al (2010), and by Viola (2010) study instead the complexity of generating -or sampling -certain distributions. Viola (2010) raises the problem of exhibiting any explicit boolean function b : {0, 1} n → {0, 1} such that no small, unbounded fanin constant-depth circuit (i.e., AC 0 ) can generate the distribution (x, b(x)) given random bits.…”