Distributed Community Prediction for Social Graphs Based on Louvain Algorithm

Abstract

Nowadays, the problem of community detection has become more and more challenging. With application in a wide range of fields such as sociology, digital marketing, bio-informatics, chemical engineering and computer science, the need for scalable and efficient solutions is strongly underlined. Especially, in the rapidly developed and widespread area of social media where the size of the corresponding networks exceeds the hundreds of millions of vertices in the average case. However, the standard sequential algorithms applications have practically proven not only infeasible but also terribly unscalable due to the excessive computation demands and the overdone resources prerequisites. Therefore, the introduction of compatible distributed machine learning solutions seems the most promising option to tackle this NP-hard class problem. The purpose of this work is to propose a novel distributed community detection methodology, based on the supervised community prediction concept that is extremely scalable, remarkably efficient and circumvent the intrinsic adversities of classic community detection approaches.

A Distributed Bagging Ensemble Methodology for Community Prediction in Social Networks

“…• the community prediction methodology requiring the extraction of a representative subgraph [4], which will be now referred as the representative subgraph's community prediction methodology in terms of brevity, • the distributed implementation of the Louvain's community detection algorithm [10,22] and • the NetworkX's implementation of the Clauset-Newman-Moore's community detection algorithm [3,25].…”

“…Due to the profitable application of community detection in plenteous scientific areas, an affluent set of algorithms [2][3][4][5][6][7][8][9][10] has already been published to tackle this a NP-hard class problem. From methods that are exclusively based on the repetitive calculation of a global network topology metric, to alternatives inspired by discrete mathematics and physics, the pluralism of classic community detection processes is indeed remarkable.…”

“…The primary intention of community prediction [4] was the introduction of an approximation methodology that by properly training a distributed learner to behave similarly to a classic community detection algorithm, having the additional advantage of stunning scalability and eminent efficiency, would feasibly proceed to the qualitative disclosing of the underlying community structure of big data social networks. Therefore, the concept of community prediction is practically a synonym for approximately detecting/predicting the subjacent community structure of any given information network based on its rudimentary network topology measures, regardless of its size.…”

“…Nevertheless, by concentrating on social graph's context, a community can intuitively be defined as the group of vertices which are more densely connected with each other (a.k.a., intra-connected) than connected with the rest of the graph (a.k.a., inter-connected). Thus, community detection can alternatively be interpreted as the edges' classification to either intra-connection, which are the edges linking vertices of the same community, or inter-connection that are the edges linking vertices of different communities [3,4].As thoroughly described in [2-10], due to community detection's widespread application, ample research has already been conducted to efficiently unveil the information networks' subjacent community structure. The classic community detection algorithms are originally designed to be generally applied to any information network.…”

“…Nevertheless, by concentrating on social graph's context, a community can intuitively be defined as the group of vertices which are more densely connected with each other (a.k.a., intra-connected) than connected with the rest of the graph (a.k.a., inter-connected). Thus, community detection can alternatively be interpreted as the edges' classification to either intra-connection, which are the edges linking vertices of the same community, or inter-connection that are the edges linking vertices of different communities [3,4].…”