RGraph: Asynchronous graph processing based on asymmetry of remote direct memory access

Abstract

The scale of real-world graphs is constantly growing. To deal with large-scale graphs, distributed graph processing has attracted much research efforts. Existing distributed graph processing systems are commonly built on traditional TCP/IP communication stack, which leads to network bottleneck because of low bandwidth and heavy kernel stack operations. Meanwhile, in real power-law graphs, the average number of mirror vertices after graph partitioning is very large, resulting in significant communication overhead among nodes. The emerging high-performance Remote Direct Memory Access (RDMA) network has the features of low latency, high bandwidth, and low CPU overhead, which brings new opportunities for distributed graph processing systems. Existing RDMA-assisted graph processing systems focus on synchronous execution, which imposes barriers between consecutive iterations. Synchronous execution transfers bulk data among nodes and thus only needs a small number of network transfers. However, synchronous execution is usually less efficient than asynchronous execution because of bulk synchronization. Asynchronous execution accelerates graph processing by eliminating barriers, which in turn requires to transfer a large amount of small size data. In this paper, we propose RGraph, an RDMA-assisted asynchronous distributed graph processing system. RGraph distributes edges into two parts to isolate master and mirror vertices. RGraph exploits the asymmetry of RDMA to accelerate the one-tomany communication between master and mirror vertices. We implement RGraph on top of PowerGraph and conduct comprehensive experiments with large-scale real graphs to evaluate its performance. Results show that compared to existing designs, RGraph reduces the execution time by up to 81%.