Realtime index-free single source SimRank processing on web-scale graphs

Abstract

Given a graph G and a node u ∈ G, a single source Sim-Rank query evaluates the similarity between u and every node v ∈ G. Existing approaches to single source SimRank computation incur either long query response time, or expensive pre-computation, which needs to be performed again whenever the graph G changes. Consequently, to our knowledge none of them is ideal for scenarios in which (i) query processing must be done in realtime, and (ii) the underlying graph G is massive, with frequent updates.Motivated by this, we propose SimPush, a novel algorithm that answers single source SimRank queries without any pre-computation, and at the same time achieves significantly higher query processing speed than even the fastest known index-based solutions. Further, SimPush provides rigorous result quality guarantees, and its high performance does not rely on any strong assumption of the underlying graph. Specifically, compared to existing methods, SimPush employs a radically different algorithmic design that focuses on (i) identifying a small number of nodes relevant to the query, and subsequently (ii) computing statistics and performing residue push from these nodes only.We prove the correctness of SimPush, analyze its time complexity, and compare its asymptotic performance with that of existing methods. Meanwhile, we evaluate the practical performance of SimPush through extensive experiments on 9 real datasets. The results demonstrate that SimPush consistently outperforms all existing solutions, often by over an order of magnitude. In particular, on a commodity machine, SimPush answers a single source SimRank query on a web graph containing over 133 million nodes and 5.4 billion edges in under 62 milliseconds, with 0.00035 empirical error, while the fastest index-based competitor needs 1.18 seconds.

EBSim: the algorithm for approximate single pair queries

“…In addition, there are also algorithms that adopt a mixed technical approach. For example, the algorithm proposed in [20] combines the local push algorithm [16] and random walk sampling from Personalized PageRank (PPR) to achieve efficient queries. For algorithms targeting single pair queries, results can be efficiently queried within milliseconds.…”

“…Based on the input and output scale of the SimRank query problem, it is generally divided into three categories: single pair queries, single source queries, and all pair queries. A number of high-quality algorithms [14,15,16,17,18,19] have emerged in these areas as well. In this paper, we mainly focus on single pair query algorithms.…”

Efficient Estimation of Pairwise Effective Resistance

“…In addition to the above similarity/dissimilarity measures, there also exist a plethora of other random walk-based measures [56,75,76] akin to ER, as surveyed in [40,78].…”

Picture semantic similarity search based on bipartite network of picture-tag type

“…UniWalk [ 30 ] calculates the similarities between objects based on Monte Carlo, which could directly locate the top k similar vertices for any single source via R sampling paths originating from the single source. SimPush [ 31 ] speeds up query processing by identifying a small number of nodes, then computes statistics and performs residue push from these nodes. These measures have been applied in numerous applications, such as spam detection [ 32 ], web page ranking [ 33 ], citation analysis [ 34 ].…”