Efficient Sampling of Dependency Structure

Unbiased and Efficient Sampling of Dependency Trees

“…With this trick the total worst-case complexity is O(n 3 ). Zmigrod et al (2021b) showed that even though Colbourn's algorithm was originally designed for unconstrained spanning trees it can easily be extended to sampling dependency trees by just using the version of Matrix-Tree Theorem by Koo et al (2007). The details of this version of MTT are presented in the Appendix A.…”

“…sample an edge e from the set of all edges that are coming out of ROOT by using their This extension is very simple, but unfortunately, as we show here, it is biased. The proof of unbiasedness present in Zmigrod et al (2021b) is incorrect because it ignores the normalization constant. This is an error because the normalization constant for G and G is not the same and therefore should influence the rest of sampling.…”

“…For instance, in situations with low entropy distributions it is very likely that standard sampling will provide many repetitions of the same set of samples and in that way be inefficient in estimating the target value. Zmigrod et al (2021b) propose SWOR algorithm that is based on modification of Colbourn's ancestral sampling algorithm (Colbourn et al, 1996). They follow the general pattern for constructing SWOR algorithms by Shi et al (2020) where after one tree is sampled, its probability mass is removed from the distribution so it cannot be sampled again.…”

“…The only previous work that deals specifically with sampling from distributions of constrained dependency trees is by Zmigrod et al (2021b). They adapt algorithms for sampling spanning trees to include the single-root constraint and provide algorithms for sampling with and without replacement.…”

“…For this reason all standard inference algorithms for spanning trees are suboptimal for modeling dependency trees. Zmigrod et al (2021b) have recently proposed algorithms for sampling with and without replacement from the single-root dependency tree distribution. In this paper we show that their fastest algorithm for sampling with replacement, Wilson-RC, is in fact producing biased samples and we provide two alternatives that are unbiased.…”