The implementation of a cloud city traffic state assessment system using a novel big data architecture

Yang, Chao-Tung; Hsu, Chih‐Yu; Yan, Yin-Zhen

doi:10.1007/s10586-017-0846-z

Cited by 13 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work related to this chapter from two closely related domains: Smart Traffic (or Intelligent Transportation Systems -ITS) and, broadly, Smart City. There have been many systems for traffic management as well as the use of big data techniques for such management systems is known [8] [9] [10]. In this chapter, we focus more on specific characteristics and techniques that are designed for solving issues related to developing countries.…”

Section: Related Workmentioning

confidence: 99%

Smarter big data analytics for traffic applications in developing countries

Pham

Minh

Truong

et al. 2019

Big Data-Enabled Internet of Things

View full text Add to dashboard Cite

Internet of Things (IoT) generate huge amount of data in real time. Utilization of such data requires appropriate data storage, analytics and computation techniques so that valuable information can be extracted and immediate actions could be taken place. In this paper, we present real time traffic applications, typical IoT-enabled big data applications in smart city context, but for unreliable and fragmented infrastructures in developing countries. We analyze a case study to show challenges that we need to address when implementing big IoT data applications in such infrastructures. Based on that we outline our key design principles and techniques. We present several examples to particularly discuss how we achieve our designs and lesson learned.the fragmented network and computational infrastructures as well as diverse types of devices with different quality, in our case, we face several challenges that need novel solutions. Research ChallengesWe will discuss challenges and requirement analysis centered on the following points:• Real time processing of data: even under limited capabilities of networks and computational resources, the data still needs to be processed in a real time manner so that decisions can be made immediately to response to the current traffic conditions. Because of this requirement, the proper solutions need to be balanced between processing time and the accuracy of results. • Noises in data: the system receives huge amount of data, but only a fraction of data is with expected accuracy and currency, due to quality of devices, noises, and network delay/reliability. This problem is severe due to the quality of IoT devices and networks in developing environments. For example, the data may come from many vehicles that are stationary, not participating in the traffic. Late arrival of data due to network problems and low quality of GPS enabled devices are also major sources of noises. The challenge here is how to separate the noise in data in real time before any calculation can take place. Furthermore, this also requires us to devise techniques to improve quality of data not in real-time but not too late for other batch analytics. • Heterogeneity of data sources: given the heterogeneity and diversity of IoT devices, GPS signals are generated from different GPS enabled devices equipped on various means of transportation such as buses, trucks, taxis, mobile users on cars or motorcycles, and so on. Each means of transportation has different traveling behaviors, rate of noises, and quality of data. Therefore, each of these sources of data needs to be processed differently. • Large amount of data: the size of data causes challenges in processing and storing. A proper storage system needs to be used so that the data can be stored and retrieved for processing at minimal delay.

show abstract

Section: Related Workmentioning

confidence: 99%

Smarter big data analytics for traffic applications in developing countries

Pham

Minh

Truong

et al. 2019

Big Data-Enabled Internet of Things

View full text Add to dashboard Cite

show abstract

“…Meanwhile, there are real-life problems, in which massive parallelization of computations on Apache Hadoop or Spark, and the use of scalable environments, like the Cloud, brought significant improvements in performance of data processing and analysis. Big data challenge was observed and solved in various works devoted to intelligent transport and smart cities [11,19,42,43,74,75,84], water monitoring [12,22,90], social networks analysis [13,14,77], multimedia processing [72,82], internet of things (IoT) [9], social media monitoring [50], Life sciences [3,31,32,44,58,69] and disease data analysis [6,45,81], telecommunication [27], and finance [2], to mention just a few. Many hot issues in various sub-fields of bioinformatics were also solved with the use of Big Data ecosystems and Cloud computing, e.g., mapping nextgeneration sequence data to the human genome and other reference genomes, for use in a variety of biological analyzes including SNP discovery, genotyping and personal genomics [65], sequence analysis and assembly [17,30,34,35,47,62], multiple alignments of DNA and RNA sequences [86,91], codon analysis with local MapReduce aggregations [63], NGS data analysis [8], phylogeny …”

Section: Related Workmentioning

confidence: 99%

Spark-IDPP: high-throughput and scalable prediction of intrinsically disordered protein regions with Spark clusters on the Cloud

2018

View full text Add to dashboard Cite

Intrinsically disorder proteins (IDPs) constitute a significant part of proteins that exist and act in cells of living organisms. IDPs play key roles in central cellular processes and some of them are closely related to various human diseases, like cancer or neurodegenerative disorders. Identification of IDPs and studying their structural characteristics have become an important part of structural bioinformatics and structural genomics. However, growing amount of genomic and protein sequences in public repositories pose a pressure on existing methods for identification of IDPs. Large volumes of protein amino acid sequences need to be analyzed in terms of propensity to form disordered regions, and this task requires novel tools and scalable platforms to cope with this big biological data challenge. In this paper, we show how the identification of disordered regions of 3D protein structures can be efficiently accelerated with the use of Apache Spark cluster established and scaled on the public Cloud. For this purpose, we propose Spark-based meta-predictor (Spark-IDPP), which enables efficient prediction of disordered regions of proteins on a large-scale. Results of our performance tests show that, for large data sets, our method achieves almost linear speedup, when scaling out the computations on the 32-node Spark cluster located in the Azure cloud. This proves that through appropriate partitioning of data and by increasing the degree of parallelism, we can significantly improve efficiency of IDP predictions. Additionally, by using several basic predictors, aggregating their ranks in various consensus modes, and filtering the final outcome with a dedicated fuzzy filter, the Spark-IDPP increases the quality of predictions.

show abstract

“…In recent years, the concept of the Internet of things (IoT) has developed rapidly and has been applied successfully in different fields, including city applications [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ], medical applications [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ], industrial applications [ 19 , 20 , 21 , 22 ], agricultural applications [ 23 , 24 , 25 , 26 , 27 ], and forestry applications [ 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. The applications using IoT technology in forestry can be classified into three major aspects: forest resource planning and environmental monitoring, the intelligent management of forest fires, and the prevention of illegal logging.…”

Section: Introductionmentioning

confidence: 99%

Realization of Forest Internet of Things Using Wireless Network Communication Technology of Low-Power Wide-Area Network

Zhao

Hsu

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

This work implements an intelligent forest monitoring system using the Internet of things (IoT) with the wireless network communication technology of a low-power wide-area network (LPWAN), a long range (LoRa), and a narrow-band Internet of things (NB-IoT). A solar micro-weather station with LoRa-based sensors and communications was built to monitor the forest status and information such as the light intensity, air pressure, ultraviolet intensity, CO2, etc. Moreover, a multi-hop algorithm for the LoRa-based sensors and communications is proposed to solve the problem of long-distance communication without 3G/4G. For the forest without electricity, we installed solar panels to supply electricity for the sensors and other equipment. In order to avoid the problem of insufficient solar panels due to insufficient sunlight in the forest, we also connected each solar panel to a battery to store electricity. The experimental results show the implementation of the proposed method and its performance.

show abstract

The implementation of a cloud city traffic state assessment system using a novel big data architecture

Cited by 13 publications

References 16 publications

Smarter big data analytics for traffic applications in developing countries

Smarter big data analytics for traffic applications in developing countries

Spark-IDPP: high-throughput and scalable prediction of intrinsically disordered protein regions with Spark clusters on the Cloud

Realization of Forest Internet of Things Using Wireless Network Communication Technology of Low-Power Wide-Area Network

Contact Info

Product

Resources

About