Lower bounds for Edit Distance and Product Metrics via Poincaré-Type Inequalities

Abstract

We prove that any sketching protocol for edit distance achieving a constant approximation requires nearly logarithmic (in the strings' length) communication complexity. This is an exponential improvement over the previous, doubly-logarithmic, lower bound of [AndoniKrauthgamer, FOCS'07]. Our lower bound also applies to the Ulam distance (edit distance over nonrepetitive strings). In this special case, it is polynomially related to the recent upper bound of [AndoniIndyk-Krauthgamer, SODA'09].From a technical perspective, we prove a direct-sum theorem for sketching product metrics that is of independent interest. We show that, for any metric X that requires sketch size which is a sufficiently large constant, sketching the max-product metric d ∞ (X) requires Ω(d) bits. The conclusion, in fact, also holds for arbitrary two-way communication. The proof uses a novel technique for information complexity based on Poincaré inequalities and suggests an intimate connection between non-embeddability, sketching and communication complexity.

The Streaming Complexity of Cycle Counting, Sorting By Reversals, and Other Problems

“…Some classical streaming problems are for example estimating frequency moments, finding approximate quantiles, and other statistics of data-sets (see [26] for more). In recent years, research has increasingly focused on stream-ing algorithms for estimating complex distance metrics between strings, such as earth mover distance [2], edit distance [4,5,3], and others. These are interesting both because of their relevance in applications, and because they are so challenging: they provide good testing grounds for exploring current techniques and coming up with new ones, both for upper bounds and for lower bounds.…”

“…[23,10,11]), and the Set Disjointness problem [9,1]. For problems of estimating complicated metrics such as those discussed above, lower bound techniques seem to be much more complicated, requiring embedding arguments [3] and direct sum arguments, among others. Naturally, progress on lower bounds has been slow for those problems, and the lower bounds proved are typically logarithmic, not polynomial.…”

Sketching and Embedding are Equivalent for Norms

“…We now show some connections between these notions. A crucial tool in our result is a theorem of [AJP10], which is itself based on the tools from [BJKS02]. The latter shows a lower bound for private-coin protocols with finitely many coins.…”

“…The latter shows a lower bound for private-coin protocols with finitely many coins. We show next how the lower bounds from [AJP10,BJKS02] extend to the most general type, public-coin protocols with countably many coins.…”

“…We now state a slight modification of the theorem from [AJP10]. We will actually use its contrapositive, to conclude the absence of Poincaré inequalities.…”