An efficient graph data processing system for large‐scale social network service applications

Abstract

Summary Trust in social network draws more and more attentions from both the academia and industry fields. Public opinion analysis is a direct way to increase the trust in social network. Because the public opinion analysis can be expressed naturally by the graph algorithm and graph data are the default data organization mechanism used in large‐scale social network service applications, more and more research works apply the graph processing system to deal with the public opinion analysis. As the data volume is growing rapidly, the distributed graph systems are introduced to process the large‐scale public opinion analysis. Most of graph algorithms introduce a large number of data iterations, so the synchronization requirements between successive iterations can severely jeopardize the effectiveness of parallel operations, which makes the data aggregation and analysis operations become slower. In this paper, we propose a large‐scale graph data processing system to address these issues, which includes a graph data processing model, Arbor. Arbor develops a new graph data organization format to represent the social relationship, and the format can not only save storage space but also accelerate graph data processing operations. Furthermore, Arbor substitutes time‐constrained synchronization operations with non‐time‐constrained control message transmissions to increase the degree of parallelism. Based on the system, we put forward two most frequently used graph applications on Arbor: shortest path and PageRank. In order to evaluate the system, we compare Arbor with the other graph processing systems using large‐scale experimental graph data, and the results show that it outperforms the state‐of‐the‐art systems. Copyright © 2014 John Wiley & Sons, Ltd.

Security and reliability in big data

“…We encourage the readers to review to gain insight into the breadth and depth of problems and innovative solutions about high performance and reliable computing in the context of big data, such as a new platform for ubiquitous computing , a new application of soft computing for financial option pricing , and a new graph data processing system .…”