Improved methods to compare distance metrics in networks using uniform random spanning trees (DIMECOST)

Abstract

We consider the network analytics problem of comparing two distance metrics on the same set of n entities. The classical solution to this problem is the Mantel test, which uses permutation testing to accept or reject the null hypothesis that there is “no relationship between the two metrics.” Its computational complexity is n2 times the number of permutations (based on a user supplied parameter). This work makes two contributions: (1) DIMECOSTP, a more efficient hypothesis test based on uniform random spanning trees whose complexity is n times the number of permutations. (2) DIMECOSTCC, which uses the correlation coefficient between the two sets of edge weights in a random spanning tree as an indication of the strength of the relationship between the two distance metrics. Both methods utilize sound statistical principles. Experimental results confirm the efficacy of our methods.

DC-RST: A Parallel Algorithm for Random Spanning Trees in Network Analytics

“…As before, while the CIs using DC-RST differ slightly from those computed using Wilson's algorithm, they still contain the r value calculated by the Mantel Test. This test previously showed that simply choosing random edges from the complete graph does not suffice, as the CIs generated in this example with random edges do not contain the r value calculated by the Mantel Test [2]. [4] The Lp-norm for p ≥ 1 of a vector ⃗ x is a commonly used measure of "distance" in machine learning for clustering, and is defined by To try and address this, we tried implementing step (2) of random-partition() with a parallel shuffling algorithm instead: MergeShuffle [16], which was chosen because of the availability of an implementation that uses OpenMP.…”

“…Bourbour et al proposed an algorithm called Dimecost, which uses uniform, random spanning trees instead of matrices [2]. Recall that the distance matrix is equivalent to a weighted, undirected, complete graph on n vertices.…”

“…Proof When partitioning, there are two general "types" of edges [2] : 1 Edges within a partition (called "within" edges). Formally: given a subset of vertices 2 Edges spanning between partitions (called "between" edges). Formally: given two subsets of vertices…”

“…1 Edges within a partition (called "within" edges). Formally: given a subset of vertices 2 Edges spanning between partitions (called "between" edges). Formally: given two subsets of vertices…”

“…recently presented DimeCost, similar to the Mantel Test, that utilizes uniform random spanning trees. DimeCost has an improved complexity of Θ(t • n) [2].…”