No abstract

We show that a large fraction of the data-structure lower bounds known today in fact follow by reduction from the communication complexity of lopsided (asymmetric) set disjointness. This includes lower bounds for:• high-dimensional problems, where the goal is to show large space lower bounds.• constant-dimensional geometric problems, where the goal is to bound the query time for space O(n · polylogn).• dynamic problems, where we are looking for a trade-off between query and update time. (In this case, our bounds are slightly weaker than the originals, losing a lg lg n factor.)Our reductions also imply the following new results:• an Ω(lg n/ lg lg n) bound for 4-dimensional range reporting, given space O(n · polylogn). This is quite timely, since a recent result [39] solved 3D reporting in O(lg 2 lg n) time, raising the prospect that higher dimensions could also be easy.• a tight space lower bound for the partial match problem, for constant query time.• the first lower bound for reachability oracles.In the process, we prove optimal randomized lower bounds for lopsided set disjointness. * The conference version of this paper appeared in FOCS'08 under the title (Data) Structures. † mip@alum.mit.edu. AT&T Labs. Parts of this work were done while the author was at MIT and IBM Almaden Research Center.

We convert cell-probe lower bounds for polynomial space into stronger lower bounds for near-linear space. Our technique applies to any lower bound proved through the richness method. For example, it applies to partial match, and to near-neighbor problems, either for randomized exact search, or for deterministic approximate search (which are thought to exhibit the curse of dimensionality). These problems are motivated by search in large databases, so near-linear space is the most relevant regime.Typically, richness has been used to imply Ω(d/ lg n) lower bounds for polynomial-space data structures, where d is the number of bits of a query. This is the highest lower bound provable through the classic reduction to communication complexity. However, for space n lg O(1) n, we now obtain bounds of Ω(d/ lg d). This is a significant improvement for natural values of d, such as lg O (1) n. In the most important case of d = Θ(lg n), we have the first superconstant lower bound. From a complexity theoretic perspective, our lower bounds are the highest known for any static data structure problem, significantly improving on previous records.