An overview of Hadoop applications in transportation big data

Ma, Changxi; Zhao, Mingxi; Zhao, Yongpeng

doi:10.1016/j.jtte.2023.05.003

Cited by 7 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A distributed framework concept large distributed computing cluster can be formed that can be installed on a low-commodity hardware device. The HDFS and MapReduce programming models are the two parts that make up the Hadoop framework [23][24][25]. MapReduce framework is based on the parallel processing concept for handling massive data sets [23].…”

Section: Hadoop Distributed Environmentmentioning

confidence: 99%

Hybrid Encryption for Securing HDFS Data in Big Data Environments

shivani

2024

Preprint

View full text Add to dashboard Cite

In the previous era, Conventional encryption schemes that work on file were linked with a lot of processing overhead that negatively impacted performance in the Hadoop framework. But now, With the help of the MapReduce framework or parallel processing method, this paper suggests a unique solution for securing large data and boosting performance: A hybrid encryption technique that combines the Twofish and AES algorithms with the map-reduce framework inside the HDFS (Hadoop Distributed File System) environment. Different conventional methods exist like Twofish + RSA, and AES + Twofish as hybrid encryption that is not decent for dealing with large data, our solution drastically improves performance through parallel encryption by mapper-reducer processes and advances the security of HDFS storage data. In this paper, we use hybrid encryption using AES with Twofish because Twofish provides better data security, and AES is used for optimizing the speed of data encryption, and decryption. Empirical findings validate the suggested approach's effectiveness in protecting private information kept in HDFS and improving performance in terms of speed of encryption ≅ 2–3%, throughput ≅1–2%, and efficiency ≅ 1–2% parameters. This work enhances the performance and reduces the risks associated with unwanted access to important data assets by data security in HDFS-based systems.

show abstract

Section: Hadoop Distributed Environmentmentioning

confidence: 99%

Hybrid Encryption for Securing HDFS Data in Big Data Environments

shivani

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…MapReduce's applications, leveraging its features and benefits [70], span data mining and extraction for reports [71], big-data graphical computation [72], machine learning challenges [73], statistical machine translation [74], spam detection [75] satellite image data processing [76], and problem clustering [77], among others. MapReduce operates through a combination of map and reduce functions, which together handle machine failures, parallelize computations across vast clusters, and facilitate inter-machine communication scheduling [78].…”

Section: Mapreducementioning

confidence: 99%

Streamline Intelligent Crowd Monitoring with IoT Cloud Computing Middleware

Gazis,

Katsiri

2024

Sensors

View full text Add to dashboard Cite

This article introduces a novel middleware that utilizes cost-effective, low-power computing devices like Raspberry Pi to analyze data from wireless sensor networks (WSNs). It is designed for indoor settings like historical buildings and museums, tracking visitors and identifying points of interest. It serves as an evacuation aid by monitoring occupancy and gauging the popularity of specific areas, subjects, or art exhibitions. The middleware employs a basic form of the MapReduce algorithm to gather WSN data and distribute it across available computer nodes. Data collected by RFID sensors on visitor badges is stored on mini-computers placed in exhibition rooms and then transmitted to a remote database after a preset time frame. Utilizing MapReduce for data analysis and a leader election algorithm for fault tolerance, this middleware showcases its viability through metrics, demonstrating applications like swift prototyping and accurate validation of findings. Despite using simpler hardware, its performance matches resource-intensive methods involving audiovisual and AI techniques. This design’s innovation lies in its fault-tolerant, distributed setup using budget-friendly, low-power devices rather than resource-heavy hardware or methods. Successfully tested at a historical building in Greece (M. Hatzidakis’ residence), it is tailored for indoor spaces. This paper compares its algorithmic application layer with other implementations, highlighting its technical strengths and advantages. Particularly relevant in the wake of the COVID-19 pandemic and general monitoring middleware for indoor locations, this middleware holds promise in tracking visitor counts and overall building occupancy.

show abstract

“…Under the heading "Leveraging Big Data Analytics for Urban Planning: A Study Using the Big Data Analytics Efficiency Test," this research study sets out to investigate the significant ramifications of big data analytics with regard to urban development and planning. Critical urban concerns including transportation, housing, resource management, and environmental sustainability need new solutions due to the city's fast urbanization and the difficulties of managing big and dynamic populations [6]- [10]. Effective big data analytics application emerges as a gamechanging option in this dynamic urban environment, enabling cities to make data-driven choices, maximize resource allocation, and improve the standard of living for their citizens.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Big Data Analytics for Urban Planning: A Study Using the Big Data Analytics Efficiency Test

Elena,

Singh,

Sobti

et al. 2024

BIO Web Conf.

View full text Add to dashboard Cite

Data from a variety of sample cities was evaluated as part of a research looking into the integration of big data analytics into urban planning. The goals were to evaluate the impact of data analytics infrastructure, data volume and processing time, urban development initiatives, and data analytics efficiency. The results showed significant differences in data analytics resources across cities, indicating different levels of investment and preparedness for data-driven decision making. It was clear that cities could handle large amounts of data efficiently thanks to their strong data processing skills. Data analytics have an impact on urban development initiatives, highlighting the revolutionary potential of data-driven urban planning. The outcomes of efficiency tests demonstrated how data analytics procedures are useful for improving urban services and for making well-informed judgments. This study offers important new insights into the mechanics of data-driven urban planning and how it can influence how cities develop in the future.

show abstract

An overview of Hadoop applications in transportation big data

Cited by 7 publications

References 32 publications

Hybrid Encryption for Securing HDFS Data in Big Data Environments

Hybrid Encryption for Securing HDFS Data in Big Data Environments

Streamline Intelligent Crowd Monitoring with IoT Cloud Computing Middleware

Leveraging Big Data Analytics for Urban Planning: A Study Using the Big Data Analytics Efficiency Test

Contact Info

Product

Resources

About