Efficient B-tree based indexing for cloud data processing

Abstract

A Cloud may be seen as a type of flexible computing infrastructure consisting of many compute nodes, where resizable computing capacities can be provided to different customers. To fully harness the power of the Cloud, efficient data management is needed to handle huge volumes of data and support a large number of concurrent end users. To achieve that, a scalable and high-throughput indexing scheme is generally required. Such an indexing scheme must not only incur a low maintenance cost but also support parallel search to improve scalability. In this paper, we present a novel, scalable B + -tree based indexing scheme for efficient data processing in the Cloud. Our approach can be summarized as follows. First, we build a local B + -tree index for each compute node which only indexes data residing on the node. Second, we organize the compute nodes as a structured overlay and publish a portion of the local B + -tree nodes to the overlay for efficient query processing. Finally, we propose an adaptive algorithm to select the published B + -tree nodes according to query patterns. We conduct extensive experiments on Amazon's EC2, and the results demonstrate that our indexing scheme is dynamic, efficient and scalable.

Dynamic multidimensional index for large-scale cloud data

“…Several studies [15][16][17][18][19][20][21][22][23] focusing on efficient indexes in cloud storage systems have been conducted. The study in [15] proposed a Trojan index to improve runtime performance.…”

“…However, it consumes considerable memory space to cache index information in the client, and it is unsuitable for processing multidimensional queries. The studies in [18,19] proposed an improved B+ tree index. This solution adopt a doublelayer index framework.…”

A distributed tree data structure for real-time OLAP on cloud architectures

“…However, none of these methods provides real-time OLAP functionality. There are various publications on distributed B-trees for cloud platforms such as [29]. However, these method only supports 1-dimensional indices which are insufficient for OLAP queries.…”

An Efficient Distributed B-tree Index Method in Cloud Computing

“…Therefore, the hash function becomes the main index of data, and the required data can be quickly accessed according to the hash value of keys [5,6,7]. However, in addition to the data query via keys, users also turn to other properties for point search or range search [8]. For example, in an online video system (such as Youtube [9]), each video contains a variety of information, including video ID, program name, upload time, times of plays.…”

“…At present, in the cloud computing environment, inverted index, the commonly used secondary index, can scan all storage nodes by multiple MapReduce [10] processes and generate inverted files. Inverted index is an off-line batch process, and it cannot realize timely query of newly inserted data [8]. For example, the record inserted into Google Base cannot be accessed by users until it is re-indexed next time (maybe one day later).…”