Multi-resolution Social Network Community Identification and Maintenance on Big Data Platform

Abstract

Abstract-Community identification in social networks is of great interest and with dynamic changes to its graph representation and content, the incremental maintenance of community poses significant challenges in computation. Moreover, the intensity of community engagement can be distinguished at multiple levels, resulting in a multi-resolution community representation that has to be maintained over time. In this paper, we first formalize this problem using the k-core metric projected at multiple k values, so that multiple community resolutions are represented with multiple k-core graphs. We then present distributed algorithms to construct and maintain a multi-k-core graph, implemented on the scalable big-data platform Apache HBase. Our experimental evaluation results demonstrate orders of magnitude speedup by maintaining multi-k-core incrementally over complete reconstruction. Our algorithms thus enable practitioners to create and maintain communities at multiple resolutions on different topics in rich social network content simultaneously.

Efficient community identification and maintenance at multiple resolutions on distributed datastores

“…We implemented, tested, and analyzed our algorithms on the open-source Hadoop HBase Big Data processing framework. Therefore, before going into the details of our proposed algorithms, readers are encouraged to read the preliminary version of our paper [1] or some other reference about HBase to become familiar with the Hadoop Big Data programming framework where our distributed k-core algorithms are implemented. …”

Graph Aware Caching Policy for Distributed Graph Stores

“…The implementation details of these system and some use cases are described in [14], [15], [16]. While running our graph algorithms on top of this platform, we experienced that a substantial amount of memory on our servers is available for use.…”