Testing monotone high-dimensional distributions

Abstract

A monotone distribution P over a (partially) ordered domain assigns higher probability to y than to x if y ≥ x in the order. We study several natural problems concerning testing properties of monotone distributions over the n-dimensional Boolean cube, given access to random draws from the distribution being tested. We give a poly(n)-time algorithm for testing whether a monotone distribution is equivalent to or -far (in the L 1 norm) from the uniform distribution. A key ingredient of the algorithm is a generalization of a known isoperimetric inequality for the Boolean cube. We also introduce a method for proving lower bounds on various problems of testing monotone distributions over the n-dimensional Boolean cube, based on a new decomposition technique for monotone distributions. We use this method to show that our uniformity testing algorithm is optimal up to polylog(n) factors, and also to give exponential lower bounds on the complexity of several other problems, including testing whether a monotone distribution is identical to or -far from a fixed known monotone product distribution and approximating the entropy of an unknown monotone distribution.

Invariance in Property Testing

“…A better approximation in time (| |) is also not too hard (see [48]). We note that the recent results in testing properties of distributions [30,10,8,9,11,1,52,50,55] have revealed many properties that can be tested in (| |) samples, and in several cases given nearly-tight bounds (to within | | (1) factors) on the query complexity of such tests.…”

Efficient compression of monotone and m-modal distributions

“…The sample complexity of learning m-modal distributions over an alphabet of size k was considered by [21]. Testing distributions for monotonicity has been considered in varied settings [22][23][24][25][26].…”

Measuring independence of datasets

“…Statistical distance is one of the most fundamental metrics for measuring the similarity of two distributions, and it has been a metric of choice in many papers that discuss distribution closeness (see e.g., [1,2,4,6,21,28,27]). Given two distributions over a discrete domain, the statistical distance is half of L 1 distance between the probability vectors.…”