MapReduce Preprocess of Big Graphs for Rapid Connected Components Detection

Abdolazimi, Reyhaneh; Heidari, Maryam; Esmaeilzadeh, Armin; Naderi, Hassan

doi:10.1109/ccwc54503.2022.9720798

Cited by 16 publications

(2 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many studies have developed methods for quickly and accurately analyzing graph streams, in which numerous changes occur in real time [9][10][11]. Connected component (CC) detection is one of the most basic algorithms used in graph analysis [12][13][14][15][16]. A CC must have paths that can connect all vertices to each other in an undirected graph and must not have paths connected to vertices belonging to other CCs.…”

Section: Introductionmentioning

confidence: 99%

Incremental Connected Component Detection for Graph Streams on GPU

et al. 2023

View full text Add to dashboard Cite

Studies on the real-time detection of connected components in graph streams have been carried out. The existing connected component detection method cannot process connected components incrementally, and the performance deteriorates due to frequent data transmission when GPU is used. In this paper, we propose a new incremental processing method to solve the problems found in the existing methods for detecting connected components on GPUs. The proposed method minimizes the amount of data to be sent to the GPU by determining the subgraph affected by the graph stream update and by detecting the part to be recalculated. We consider the number of vertices to quickly determine the connected components of a graph stream on the GPU. An asynchronous execution method is used to shorten the transfer time between the CPU and the GPU according to real-time graph stream changes. In order to show that the proposed method provides fast incremental connected component detection on the GPU, we evaluated its performance using various datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

Incremental Connected Component Detection for Graph Streams on GPU

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Batch human data analysis adopts batch processing mode, first data storage, and then the static storage data set. At present, the most common mass human data computing framework is Hadoop [11][12][13], which mainly includes HDFS (Hadoop distributed file system) and MapReduce [14][15][16], which are responsible for storing static data, while the latter completes the allocation of computing tasks. Each data node gets the corresponding computing tasks, respectively, and all nodes cooperate for data analysis and value discovery.…”

Section: Introductionmentioning

confidence: 99%

Judgment of Social Governance System Risk Based on Big Data Analysis

Yang

2022

Mobile Information Systems

View full text Add to dashboard Cite

Accurate risk identification, scientific risk assessment, effective risk early warning, and real-time risk monitoring will all benefit from the advancement of data science and the development of the data industry. Simultaneously, social risk prevention and control face numerous obstacles in the growth of information technology, data integration and mining, data information disclosure, and data culture popularization in the era of big data. It can be improved in terms of risk prevention and control concept innovation. In addition, it can also be improved in terms of network public opinion guidance, control mechanism improvement, social security prevention, control system optimization, and the introduction and training of big data talent to promote scientific and accurate social risk prevention and control. As society moves into the 5.0 era, many social risks rise and new features emerge, posing larger threats and unprecedented challenges to social risk governance. Big data, as a result of the emerging technology revolution, offers a wealth of governance resources and brand-new governance concepts that traditional governance mechanisms lack, offering technical assistance. This article presents a parallel community partition algorithm using MapReduce to analyze big data. The proposed model is established for the big data thinking paradigm to promote social risk governance innovation. The proposed model supports the top-level model and does inclusive forecasting for social risk management that supports resource integration. The proposed model also realizes diversified and coordinated risk governance. The results reveal the significance of the proposed model.

show abstract

An energy‐aware resource management method in cloud‐based Internet of Things using a multi‐objective algorithm and crowding distance

Mohammed

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Internet of Things (IoT) is predicted to permeate all areas of the physical world, particularly homes and urban settings, in the next years. Cloud‐based IoT is a network of things that can be managed and inspected to create various intelligent systems over the internet. The primary technological difficulty in service computing is swiftly integrating diverse services to serve cross‐organizational business activities. It is one of the famous NP‐hard problems; therefore, this study proposes a novel service composition technique termed multiobjective particle swarm optimization and crowding distance (MOPSO‐CD) approach to solve this problem. The main issue with the MOPSO method is that the search is conducted very quickly, resulting in an incorrect response. To address this issue, we integrate MOPSO with the CD approach to provide an efficient composition service in cloud‐based IoT. The proposed method is simulated using Matlab, and the performance is compared against the performance of three other multi‐objective algorithms. The findings revealed that the proposed method outperforms different algorithms regarding availability, reliability, response time, latency, and energy consumption.

show abstract

MapReduce Preprocess of Big Graphs for Rapid Connected Components Detection

Cited by 16 publications

References 46 publications

Incremental Connected Component Detection for Graph Streams on GPU

Incremental Connected Component Detection for Graph Streams on GPU

Judgment of Social Governance System Risk Based on Big Data Analysis

An energy‐aware resource management method in cloud‐based Internet of Things using a multi‐objective algorithm and crowding distance

Contact Info

Product

Resources

About