Multi-resolution bitmap indexes for scientific data

Sinha, Rishi Rakesh; Winslett, Marianne

doi:10.1145/1272743.1272746

Cited by 75 publications

(31 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sinha and Winslet [31] successfully demonstrate parallelizable strategies for binning and encoding bitmap indexes, compressing bitmap vectors, and answering selection queries with compressed bitmap vectors. The content of their work focuses on supporting bitmap use in a highly parallel environment of multiple loosely-coupled, shared-nothing systems.…”

Section: Related Bitmap Index Workmentioning

confidence: 99%

“…For this reason, our new Data Parallel Bin-based Indexing Strategy (DP-BIS) follows the general structure of a binned bitmap index. Unfortunately, bitmap compression strategies, even the parallelizable strategies of Sinha and Winslet [31], do not support enough concurrency to take advantage of the thread-level parallelism offered by tightly-coupled architectures like GPUs. Thus one of the first objectives in our work is to develop a compression strategy, based upon the binning techniques of the binned bitmap index, that supports high levels of concurrency and reduces the amount of data required to answer a query.…”

Section: Related Bitmap Index Workmentioning

confidence: 99%

See 1 more Smart Citation

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Gosink

Bethel

et al. 2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The multi-core trend in CPUs and general purpose graphics processing units (GPUs) offers new opportunities for the database community. The increase of cores at exponential rates is likely to affect virtually every server and client in the coming decade, and presents database management systems with a huge, compelling disruption that will radically change how processing is done. This paper presents a new parallel indexing data structure for answering queries that takes full advantage of the increasing thread-level parallelism emerging in multi-core architectures. In our approach, our Data Parallel Bin-based Index Strategy (DP-BIS) first bins the base data, and then partitions and stores the values in each bin as a separate, bin-based data cluster. In answering a query, the procedures for examining the bin numbers and the bin-based data clusters offer the maximum possible level of concurrency; each record is evaluated by a single thread and all threads are processed simultaneously in parallel.We implement and demonstrate the effectiveness of DP-BIS on two multicore architectures: a multi-core CPU and a GPU. The concurrency afforded by DP-BIS allows us to fully utilize the thread-level parallelism provided by each architecture-for example, our GPU-based DP-BIS implementation simultaneously evaluates over 12,000 records with an equivalent number of concurrently executing threads. In comparing DP-BIS's performance across these architectures, we show that the GPU-based DP-BIS implementation requires significantly less computation time to answer a query than the CPU-based implementation. We also demonstrate in our analysis that DP-BIS provides better overall performance than the commonly utilized CPU and GPU-based projection index. Finally, due to data encoding, we show that DP-BIS accesses significantly smaller amounts of data than index strategies that operate solely on a column's base data; this smaller data footprint is critical for parallel processors that possess limited memory resources (e.g. GPUs).

show abstract

Section: Related Bitmap Index Workmentioning

confidence: 99%

Section: Related Bitmap Index Workmentioning

confidence: 99%

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Gosink

Bethel

et al. 2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…They are designed to answer one-sided and two-sided range queries efficiently. The three basic encoding schemes can also be composed into multi-level and multi-component encodings [6,31,42]. One well-known example of a multi-component encoding is the binary encoding scheme [39,23], where the jth bitmap of the index represents the value 2 j .…”

Section: Indexing Techniques For Structured Datamentioning

confidence: 99%

Using Bitmap Index for Joint Queries on Structured and Text Data

Stockinger

Cieslewicz

et al. 2008

Annals of Information Systems

View full text Add to dashboard Cite

The database and the information retrieval communities have been working on separate sets of techniques for querying structured data and text data, but there is a growing need to handle these types of data together. In this paper, we present a strategy to efficiently answer joint queries on both types of data. By using an efficient compression algorithm, our compressed bitmap indexes, called FastBit, are compact even when they contain millions of bitmaps. Therefore FastBit can be applied effectively on hundreds of thousands of terms over millions of documents. Bitmap indexes are designed to take advantage of data that only grows but does not change over time (append-only data), and thus are just as effective with append-only text archives. In a performance comparison against a commonly used database system with a full-text index, MySQL, we demonstrate that our indexes answer queries 50 times faster on average. Furthermore, we demonstrate that integrating FastBit with a open source database system, called MonetDB, yields similar performance gains. Since the integrated MonetDB/FastBit system provides the full SQL functionality, the overhead of supporting SQL is not the main reason for the observed performance differences. Therefore, using FastBit in other database systems can offer similar performance advantages.

show abstract

“…Bitmap indexes are known as the most effective indexing methods for conducting range queries on append-only data. Many different bitmap indexes have been proposed in the relevant literature [35,27,42,36,16,43,10,8]. Chan et al presented a general framework to study the design space of bitmap indexes for selection queries and for examining the disk-space and time characteristics that the various alternative index choices offer [8].…”

Section: Introductionmentioning

confidence: 99%

“…Yoon et al proposed a bitmap-indexing scheme for speeding up the access control for the XML documents [43]. Sinha et al proposed a multi-resolution, parallelizable bitmap index, which supports a fine-grained trade-off between storage requirements and query performance [36]. Sinha et al introduced adaptive bitmap indexes, which conform to space limits while dynamically adapting to the query load and these indexes provide excellent performance [35].…”

Section: Introductionmentioning

confidence: 99%

Developing a Dynamic Materialized View Index for Efficiently Discovering Usable Views for Progressive Queries

Zhu¹,

Zhu²,

Zuzarte³

et al. 2013

Journal of Information Processing Systems

View full text Add to dashboard Cite

Abstract-Numerous data intensive applications demand the efficient processing of a new type of query, which is called a progressive query (PQ). A PQ consists of a set of unpredictable but inter-related step-queries (SQ) that are specified by its user in a sequence of steps. A conventional DBMS was not designed to efficiently process such PQs. In our earlier work, we introduced a materialized view based approach for efficiently processing PQs, where the focus was on selecting promising views for materialization. The problem of how to efficiently find usable views from the materialized set in order to answer the SQs for a PQ remains open. In this paper, we present a new index technique, called the Dynamic Materialized View Index (DMVI), to rapidly discover usable views for answering a given SQ. The structure of the proposed index is a special ordered tree where the SQ domain tables are used as search keys and some bitmaps are kept at the leaf nodes for refined filtering. A two-level priority rule is adopted to order domain tables in the tree, which facilitates the efficient maintenance of the tree by taking into account the dynamic characteristics of various types of materialized views for PQs. The bitmap encoding methods and the strategies/algorithms to construct, search, and maintain the DMVI are suggested. The extensive experimental results demonstrate that our index technique is quite promising in improving the performance of the materialized view based query processing approach for PQs

show abstract

Multi-resolution bitmap indexes for scientific data

Cited by 75 publications

References 31 publications

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures

Using Bitmap Index for Joint Queries on Structured and Text Data

Developing a Dynamic Materialized View Index for Efficiently Discovering Usable Views for Progressive Queries

Contact Info

Product

Resources

About