Efficient Iceberg Query Evaluation Using Compressed Bitmap Index

Abstract

Decision support and knowledge discovery systems often compute aggregate values of interesting attributes by processing a huge amount of data in very large databases and/or warehouses. In particular, iceberg query is a special type of aggregation query that computes aggregate values above a user-provided threshold. Usually, only a small number of results will satisfy the threshold constraint. Yet, the results often carry very important and valuable business insights. Because of the small result set, iceberg queries offer many opportunities for deep query optimization. However, most existing iceberg query processing algorithms do not take advantage of the small-result-set property and rely heavily on the tuple-scan-based approach. This incurs intensive disk accesses and computation, resulting in long processing time especially when data size is large. Bitmap index, which builds one bitmap vector for each attribute value, is gaining popularity in both column-oriented and row-oriented databases in recent years. It occupies less space than the raw data and gives opportunities for more efficient query processing. In this paper, we exploited the property of bitmap index and developed a very effective bitmap pruning strategy for processing iceberg queries. Our index-pruning-based approach eliminates the need of scanning and processing the entire data set (table) and thus speeds up the iceberg query processing significantly. Experiments show that our approach is much more efficient than existing algorithms commonly used in row-oriented and columnoriented databases.

A Density based Priority Queue Strategy to Evaluate Iceberg Queries Efficiently using Compressed Bitmap Indices

“…Bin He et al [1], proposed an efficient vector alignment algorithm by exploiting the property of bitmap indices. This algorithm completely solves an empty bitwise-AND results problem by pruning the bitmaps whose resultants are zero vectors.…”

“…The deferred XOR strategy in GS [1] further prunes the bitmaps whose cardinality is not more than threshold after every AND operation. This is possible by deferring the XOR operations from the suite of AND operation.…”

“…[1] used compressed bitmap indices (Word Aligned Hybrid). A bitmap for an attribute in a table can be viewed as a matrix having r rows consisting corresponding number of tuples and v columns indicating the number of distinct values of an attribute.…”

Developing a Dynamic Materialized View Index for Efficiently Discovering Usable Views for Progressive Queries

“…Bitmap indexes are known as the most effective indexing methods for conducting range queries on append-only data. Many different bitmap indexes have been proposed in the relevant literature [35,27,42,36,16,43,10,8]. Chan et al presented a general framework to study the design space of bitmap indexes for selection queries and for examining the disk-space and time characteristics that the various alternative index choices offer [8].…”

“…Sinha et al introduced adaptive bitmap indexes, which conform to space limits while dynamically adapting to the query load and these indexes provide excellent performance [35]. He et al developed a bitmap pruning strategy for processing the iceberg query, which is a special type of aggregation query that computes aggregate values above a user-provided threshold [16]. Fusco et al proposed a compressed bitmap index approach that significantly reduces both CPU load and disk consumption [10].…”