Hadoop based Demography Big Data Management System

Bukhari, Syeda Sana; Park, Jinhyuck; Shin, Dong Ryeol

doi:10.1109/snpd.2018.8441032

Cited by 7 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In article [24], Brunswicker et al developed on research issue which is related to open digital collaboration and established the data analytical challenges which need to be revealed to answer these important research issues. In the paper [25], Bukhari et al proposed a method based on big data demography managing system using apache Hadoop platform to settle troubles of demography high rising data managing. In the paper [26] Malik et al have analyzed the various mechanisms of resource allocation, mode selection for underlying communications in the sense of device to device and cooperative communication techniques and they establish a new technique LTE-Advanced Pro.…”

Section: Related Workmentioning

confidence: 99%

Big data traffic management in vehicular ad-hoc network

Tantaoui¹,

Driss

Kabil³

2021

IJECE

View full text Add to dashboard Cite

<span>Today, the world has experienced a new trend with regard to data system management, traditional database management tools have become outdated and they will no longer be able to process the mass of data generated by different systems, that's why big data is there to process this mass of data to bring out crucial information hidden in this data, and without big data technologies the treatment is very difficult to manage; among the domains that uses big data technologies is vehicular ad-hoc network to manage their voluminous data. In this article, we establish in the first step a method that allow to detect anomalies or accidents within the road and compute the time spent in each road section in real time, which permit us to obtain a database having the estimated time spent in all sections in real time, this will serve us to send to the vehicles the right estimated time of arrival all along their journey and the optimal route to attain their destination. This database is useful to utilize it like inputs for machine learning to predict the places and times where the probability of accidents is higher. The experimental results prove that our method permits us to avoid congestions and apportion the load of vehicles in all roads effectively, also it contributes to road safety.</span>

show abstract

Section: Related Workmentioning

confidence: 99%

Big data traffic management in vehicular ad-hoc network

Tantaoui¹,

Driss

Kabil³

2021

IJECE

View full text Add to dashboard Cite

show abstract

“…Hadoop uses the MapReduce framework to perform the computation in two different stages: (i) the Map stage, which divides the task (computation) of the job in parts to be computed by different nodes (in parallel) and (ii) the Reduce stage, which processes and summarizes the data coming from the Map stage (different nodes) [74]. Hive is a Data warehouse infrastructure that uses SQL queries (HiveQL) to query data from Hadoop [72]. It converts the input program written in HiveQL into MapReduce jobs executed on Hadoop [75].…”

Section: Dataset Descriptionmentioning

confidence: 99%

“…Hadoop is an open-source framework that processes offline big data in a distributed manner. Hadoop is characterized for having different key components for this computation, such as the Hadoop Distributed File System (HDFS) [70], a MapReduce framework [71], the Hive warehouse [72], the Apache HBase distributed database [73], etc. HDFS is the primary storage in Hadoop and it manages the storage of data in blocks to be processed by any job [74].…”

Section: Dataset Descriptionmentioning

confidence: 99%

KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach

2021

View full text Add to dashboard Cite

High-Performance Computing (HPC) systems need to be constantly monitored to ensure their stability. The monitoring systems collect a tremendous amount of data about different parameters or Key Performance Indicators (KPIs), such as resource usage, IO waiting time, etc. A proper analysis of this data, usually stored as time series, can provide insight in choosing the right management strategies as well as the early detection of issues. In this paper, we introduce a methodology to cluster HPC jobs according to their KPI indicators. Our approach reduces the inherent high dimensionality of the collected data by applying two techniques to the time series: literature-based and variance-based feature extraction. We also define a procedure to visualize the obtained clusters by combining the two previous approaches and the Principal Component Analysis (PCA). Finally, we have validated our contributions on a real data set to conclude that those KPIs related to CPU usage provide the best cohesion and separation for clustering analysis and the good results of our visualization methodology. INDEX TERMS Clustering, feature extraction, high-performance computing, time series analysis.

show abstract

“…Apache Hadoop ecosystem turns out to be a good solution for administering big data problems. In the paper [3], H.Bukhari, S. Park, J.Shin proposed a method based on big data demography managing system using apache Hadoop platform to settle troubles of demography high rising data managing. The essential units of the proposed system are Sqoop, HBase and Hive.…”

Section: Related Workmentioning

confidence: 99%

Toward stopping spread of coronavirus with the help of Big Data density Management

Tantaoui

Kabil

Laanaoui

2020

2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4)

View full text Add to dashboard Cite

this hour, coronavirus is a threat to our life and especially to our close family and the elderly person, who have more or less poor health, it will harm human beings and economics, so we have to find a method to limit the risk of this devastating virus and to discover a way to decrease its propagation. Our concern is to compute people density on many different locations of interest in the whole city with intention of dispatching people to different areas by avoiding as much as possible congestions. The support of using big data technologies is very crucial to process data with a fast manner and in real time. Today traditional database management tools is unable to manage the voluminous data generated by different system sources, that's why big data technologies will give us its support to manage this mass of data to extract crucial information from this enormous data, and without big data technologies, the process turn out very difficult to control. In the present paper, we first propose a method that will calculate densities of people in different city's places in real time, this will allow us to prevent people locations to avoid where there are congestions and reorient them to another lightened safe locations. And then we will store in a database all human contacts that have taken place to warn people who have been in contact or who have been close to the contaminated ones.

show abstract

Hadoop based Demography Big Data Management System

Cited by 7 publications

References 6 publications

Big data traffic management in vehicular ad-hoc network

Big data traffic management in vehicular ad-hoc network

KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach

Toward stopping spread of coronavirus with the help of Big Data density Management

Contact Info

Product

Resources

About