Complexity and Geometry of Sampling Connected Graph Partitions

Abstract

In this paper, we prove intractability results about sampling from the set of partitions of a planar graph into connected components. Our proofs are motivated by a technique introduced by Jerrum, Valiant, and Vazirani. Moreover, we use gadgets inspired by their technique to provide families of graphs where the "flip walk" Markov chain used in practice for this sampling task exhibits exponentially slow mixing. Supporting our theoretical results we present some empirical evidence demonstrating the slow mixing of the flip walk on grid graphs and on real data. Inspired by connections to the statistical physics of self-avoiding walks, we investigate the sensitivity of certain popular sampling algorithms to the graph topology. Finally, we discuss a few cases where the sampling problem is tractable. Applications to political redistricting have recently brought increased attention to this problem, and we articulate open questions about this application that are highlighted by our results.

Recombination: A family of Markov chains for redistricting

“…Our experiments below support the intuition that the time needed for effective sampling has moderate growth; tens of thousands of recombination steps give stable results on practical-scale problems whether we work with the roughly 9000 precincts of Pennsylvania or the roughly 100,000 census blocks in our Virginia experiments. Note that these observations do not contradict the theoretical obstructions in [NDS19], since ReCom is not designed to target the uniform distribution or any other distribution known to be intractable. While ReCom is decidedly nonuniform, the arguments in §5.1 indicate that this nonuniformity is desirable, as the chain preferentially samples from plans that comport with traditional districting principles.…”

“…As noted above, this is often done with a standard technique in MCMC called the Metropolis-Hastings algorithm: fix a compactness score, such as a notion of boundary length |∂P |, prescribe a distribution proportional to x |∂P | on the state space, and use the Metropolis-Hastings rule to preferentially accept more compact plans. As discussed above in §5.2, there are computational obstructions to sampling proportionally to x |∂P | [NDS19]. Even if we are unable to achieve a perfect sample from this distribution, however, it could be the case that this strategy generates a suitably diverse ensemble in reasonable time for our applications.…”

“…In the study of computational complexity, P ⊆ RP ⊆ N P are complexity classes (polynomial time, randomized polynomial time, and nondeterministic polynomial time) and it is widely believed that P = RP , and RP = N P . Recent theoretical work of DeFord-Najt-Solomon [NDS19] shows that flip and uniform flip procedures mix exponentially slowly on several families of graphs, including planar triangulations of bounded degree. The authors also show that sampling proportionally to x |∂P | for any 0 < x ≤ 1 is intractable, in the sense that an efficient solution would imply RP = N P .…”

Geometry of Graph Partitions via Optimal Transport

“…Otherwise, the step is rejected and we retry. As shown in [31], this procedure results in partitions with geometrically irregular and highly non-compact components; as an example, the start and end positions of each chain are shown in Figure 8. We let both Markov chains run and compute the distance between them at every 1,000th step, using Hamming and transport distance; Figure 9 shows the result.…”

Compact Redistricting Plans Have Many Spanning Trees

“…The case for ReCom rests on its ability to generate plans consisting of compact districts with regular shapes. A practical and well-studied measure for compactness in the graph partitioning setting is to count the total number of cut edges -the edges whose endpoints lie in different districts; plans with fewer cut edges are more compact [10,22]. Empirically, ReCom does generate compact plans according to this measure.…”