A MapReduce-based approach to social network big data mining

Abstract

The rapid development of social networks has facilitated the convenience of users to receive information. As a network communication platform for people’s daily use, microblog has countless information data. In view of the low efficiency and poor clustering effect of K-means algorithm, a parallel K-means clustering algorithm based on MapReduce model is studied; In order to alleviate the difficulty in calculating the similarity of microblog topic text, the space vector model and semantic similarity are used to calculate the similarity between texts to improve the quality of microblog text classification. The data expansion rate of corresponding nodes under different data sets shows that the average expansion rate of the parallel K-means algorithm reaches 0.89, and the running rate is the highest. The results show that the parallel K-means algorithm has good clustering stability and the highest clustering quality, reaching 1.24; The clustering time of the algorithm is the shortest, the average clustering time is 1.27 minutes, and the clustering effect and efficiency of the algorithm are the best. In the quality analysis of Weibo topic recommendation, the accuracy of P-K-means recommendation is 95.64%, user satisfaction is 98.64%, and the recommendation effect is also the best. It shows that the research on the parallel K-means clustering algorithm based on MapReduce has the best performance in microblogging topic mining and recommendation, which can efficiently recommend topics of interest to users and enhance users’ microblogging experience.

Streamline Intelligent Crowd Monitoring with IoT Cloud Computing Middleware

“…MapReduce's applications, leveraging its features and benefits [70], span data mining and extraction for reports [71], big-data graphical computation [72], machine learning challenges [73], statistical machine translation [74], spam detection [75] satellite image data processing [76], and problem clustering [77], among others. MapReduce operates through a combination of map and reduce functions, which together handle machine failures, parallelize computations across vast clusters, and facilitate inter-machine communication scheduling [78].…”