Distributed Indexing Methods for Moving Objects based on Spark Stream

Abstract

Generally

Parallel Insertion and Indexing Method for Large Amount of Spatiotemporal Data Using Dynamic Multilevel Grid Technique

“…Reference [5] proposes an in-memory distributed indexing method for moving objects based on Apache Spark. The basic technique of Reference [5] is a simple gird index.…”

“…Reference [5] proposes an in-memory distributed indexing method for moving objects based on Apache Spark. The basic technique of Reference [5] is a simple gird index. Reference [5] adds new transformation operators and output operators such as bulkLoad, bulkInsert, splitIndex, search to index, and query moving objects in real-time.…”

“…The basic technique of Reference [5] is a simple gird index. Reference [5] adds new transformation operators and output operators such as bulkLoad, bulkInsert, splitIndex, search to index, and query moving objects in real-time. The input stream is the location data of moving objects that are transmitted periodically from vehicles.…”

“…Data and query collectors which are designed based on Apache Kafka receives location data and time from vehicles and queries from users. Index manager creates grid-based spatiotemporal index structures, and it is an enhanced version of that in Reference [5], which is based on Spark Streaming to consider the case of the full of main memory. Also, the indexing method of this paper provides snapshot isolation level of transactional processing with multi-version concurrency control techniques based on RDD(Resilient Distributed Dataset)s of Apache Spark.…”

“…Recently, various methods have been proposed to deal with the increased importance of very large spatiotemporal data processing. In some studies, parallel and distributed indexing methods to process the location data of moving objects have been proposed [1][2][3][4][5][6][7][8][9][10]. According to reference [11], these methods can be divided into two groups depending on what big data processing frameworks, such as Apache Hadoop [12] and Apache Spark [13], are using.…”