Uncertain, or probabilistic, graphs have been increasingly used to represent noisy linked data in many emerging applications, and have recently attracted the attention of the database research community. A fundamental problem on uncertain graphs is the s-t reliability, which measures the probability that a target node t is reachable from a source node s in a probabilistic (or uncertain) graph, i.e., a graph where every edge is assigned a probability of existence.Due to the inherent complexity of the s-t reliability estimation problem (#P-hard), various sampling and indexing based efficient algorithms were proposed in the literature. However, since they have not been thoroughly compared with each other, it is not clear whether the later algorithm outperforms the earlier ones. More importantly, the comparison framework, datasets, and metrics were often not consistent (e.g., different convergence criteria were employed to find the optimal number of samples) across these works. We address this serious concern by re-implementing six state-ofthe-art s-t reliability estimation methods in a common system and code base, using several medium and large-scale, real-world graph datasets, identical evaluation metrics, and query workloads.Through our systematic and in-depth analysis of experimental results, we report surprising findings, such as many follow-up algorithms can actually be several orders of magnitude inefficient, less accurate, and more memory intensive compared to the ones that were proposed earlier. We conclude by discussing our recommendations on the road ahead.
Reliability in uncertain graphsTheory Polynomial-time upper/lower bounds [5,7,8,16,27,35] Representative possible worlds [33,37] Simplified version of reliability problem Most reliable path [9,22,26] Algorithm Sequential Distributed [10, 47] Fundamental s-t reliability query Other queries: top-k query[43], distance-constrained query [20], conditional reliability [23], probabilistic road network query [19], influence maximization [21], etc. Uncertainty reduction: adaptive edge testing [13-15], crowdsourcing [31] Estimator Monte Carlo (MC) sampling [12] Variant of MC: Lazy propagation [30] Recursive estimator: RHH [20], RSS [28] Index scheme BFS Sharing [45] ProbTree [32]Our Focus Figure 2: The broad spectrum of reliability problem over uncertain graphs sive samplings [20,28] and shared possible worlds [45], as well as other indexing methods [32]. With the wide range of algorithms available for estimating the s-t reliability over uncertain graphs, there is an urgent need to realize their trade-offs, and to employ the best algorithm for a given scenario.As depicted in Figure 1, we find serious concerns in the existing experimental comparisons of state-of-the-art reliability estimation algorithms over uncertain graphs. (1) There is no prior work that compared all state-of-the-art methods with each other. It is, therefore, difficult to draw a general conclusion on the superiority and trade-offs of different methods.(2) As shown in Figure 1, with the exceptio...