A distributed B+Tree indexing method for processing range queries over streaming data

Abstract

A data stream exhibits as a massive unbounded sequence of data elements continuously generated at a high rate. Stream databases raise new challenges for query processing due to both the streaming nature of data which constantly changes over time and the wider range of queries submitted by the user when compared with the traditional databases. In this paper, we propose a system architecture which includes components for both distributed indexing of streaming data and distributed processing of range queries over streaming data. By exploiting the proposed system architecture, the process of indexing of streaming data and the process of querying over streaming data can be done in a distributed fashion. We also design a distributed B+Tree indexing method using the map-reduce programming model of the Apache Spark framework which creates small B+Tree indexes on the machines of a Spark cluster instead of using a large and centralized B+Tree index structure. Moreover, we propose a distributed range search algorithm to process range queries in distributed and parallel form using the set of small B+Tree indexes. By performing several experiments, we demonstrate that our proposed distributed B+Tree indexing method is scalable and efficient compared to the existing indexing methods and therefore, it can be used for applications involving data streams with a large volume of data elements and a large number of range queries.