An Efficient Method of Sharing Mass Spatio-Temporal Trajectory Data Based on Cloudera Impala for Traffic Distribution Mapping in an Urban City

Abstract

The efficient sharing of spatio-temporal trajectory data is important to understand traffic congestion in mass data. However, the data volumes of bus networks in urban cities are growing rapidly, reaching daily volumes of one hundred million datapoints. Accessing and retrieving mass spatio-temporal trajectory data in any field is hard and inefficient due to limited computational capabilities and incomplete data organization mechanisms. Therefore, we propose an optimized and efficient spatio-temporal trajectory data retrieval method based on the Cloudera Impala query engine, called ESTRI, to enhance the efficiency of mass data sharing. As an excellent query tool for mass data, Impala can be applied for mass spatio-temporal trajectory data sharing. In ESTRI we extend the spatio-temporal trajectory data retrieval function of Impala and design a suitable data partitioning method. In our experiments, the Taiyuan BeiDou (BD) bus network is selected, containing 2300 buses with BD positioning sensors, producing 20 million records every day, resulting in two difficulties as described in the Introduction section. In addition, ESTRI and MongoDB are applied in experiments. The experiments show that ESTRI achieves the most efficient data retrieval compared to retrieval using MongoDB for data volumes of fifty million, one hundred million, one hundred and fifty million, and two hundred million. The performance of ESTRI is approximately seven times higher than that of MongoDB. The experiments show that ESTRI is an effective method for retrieving mass spatio-temporal trajectory data. Finally, bus distribution mapping in Taiyuan city is achieved, describing the buses density in different regions at different times throughout the day, which can be applied in future studies of transport, such as traffic scheduling, traffic planning and traffic behavior management in intelligent public transportation systems.

Efficient Streaming Mass Spatio-Temporal Vehicle Data Access in Urban Sensor Networks Based on Apache Storm

“…The offering table stores the organization information, including the organization name, phenomenon name and sensor information, and the featureOfInterest table stores the spatial arrangement of observations, including the name and coordinates encoded within the GeoJson data type. The data tables are different from others, such as the data table in The Hadoop Distributed File System [ 33 ] and are shown as follows: UFObservation table: Stores the unfiltered observation information when the sensors finish observation, including the observation time, spatial range of the observation and the observation result. FObservation table: Stores the filtered observation information when the sensors finish observation, including the observation time, spatial range of the observation and the observation result.…”

Toplu Taşima İşletmeleri̇nde Akilli Ulaşim Si̇stem Veri̇leri̇ni̇n Büyük Veri̇ Teknoloji̇leri̇ İle Desteklenmi̇ş İş Zekâsi Mi̇mari̇si̇nde Uygulanmasi: Bi̇r Li̇teratür İncelemesi̇

“…Araştırmacılar bu ürünü koordinat verilerini almak, Taiyuan'daki trafik veri akışını haritalamak, trafik planlamasını yapmak, planlama ve davranış yönetimi gibi faaliyetler için akıllı toplu taşıma sistemlerinde bu haritalamanın kullanışlılığını arttırmak için kullanmışladır (Zhou, Chen, Yuan ve Chen, 2016). Şekil 2'de klasik ve modern CRISP-DM yaklaşımının gösterimine (veri madenciliği için çapraz endüstri standart süreci) yer verilmiştir.…”

Applying OGC Standards to Develop a Land Surveying Measurement Model

“…The Sensor Observation Service (SOS) standard that has been defined by OGC provides the specifications for the required operations and has been implemented by various programming languages and application frameworks [33]. It should be noted that considerations regarding performance issues emerge when handling mass spatio-temporal data; thus, research interest regarding the use of cutting edge technologies is under discussion (e.g., applications based on NoSQL, MongoDB or the SQL Cloudera Impala engine [34,35]). …”