Smarter big data analytics for traffic applications in developing countries

Abstract

Internet of Things (IoT) generate huge amount of data in real time. Utilization of such data requires appropriate data storage, analytics and computation techniques so that valuable information can be extracted and immediate actions could be taken place. In this paper, we present real time traffic applications, typical IoT-enabled big data applications in smart city context, but for unreliable and fragmented infrastructures in developing countries. We analyze a case study to show challenges that we need to address when implementing big IoT data applications in such infrastructures. Based on that we outline our key design principles and techniques. We present several examples to particularly discuss how we achieve our designs and lesson learned.the fragmented network and computational infrastructures as well as diverse types of devices with different quality, in our case, we face several challenges that need novel solutions. Research ChallengesWe will discuss challenges and requirement analysis centered on the following points:• Real time processing of data: even under limited capabilities of networks and computational resources, the data still needs to be processed in a real time manner so that decisions can be made immediately to response to the current traffic conditions. Because of this requirement, the proper solutions need to be balanced between processing time and the accuracy of results. • Noises in data: the system receives huge amount of data, but only a fraction of data is with expected accuracy and currency, due to quality of devices, noises, and network delay/reliability. This problem is severe due to the quality of IoT devices and networks in developing environments. For example, the data may come from many vehicles that are stationary, not participating in the traffic. Late arrival of data due to network problems and low quality of GPS enabled devices are also major sources of noises. The challenge here is how to separate the noise in data in real time before any calculation can take place. Furthermore, this also requires us to devise techniques to improve quality of data not in real-time but not too late for other batch analytics. • Heterogeneity of data sources: given the heterogeneity and diversity of IoT devices, GPS signals are generated from different GPS enabled devices equipped on various means of transportation such as buses, trucks, taxis, mobile users on cars or motorcycles, and so on. Each means of transportation has different traveling behaviors, rate of noises, and quality of data. Therefore, each of these sources of data needs to be processed differently. • Large amount of data: the size of data causes challenges in processing and storing. A proper storage system needs to be used so that the data can be stored and retrieved for processing at minimal delay.