Iceberg-cube computation with PC clusters

Ng, Raymond T.; Wagner, Alan; Yin, Yu

doi:10.1145/376284.375666

Cited by 35 publications

(38 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are divided into two main groups: work partitioning [11], [12] and data partitioning [13], [14]. In work partitioning, each processor (or node) of the cluster computes aggregates for a set of one or many views independently.…”

Section: A Rollup In Parallel Databasesmentioning

confidence: 99%

Efficient and Self-Balanced ROLLUP Aggregates for Large-Scale Data Summarization

Phan

Hoang-Xuan

Dell’Amico³

et al. 2015

2015 IEEE International Congress on Big Data

View full text Add to dashboard Cite

Abstract-Data summarization queries that compute aggregates by grouping datasets across several dimensions are essential to help users make sense of very large datasets. In this work, we focus on ROLLUP, an important operator that has been recently added to the Hadoop MapReduce ecosystem. However, its current implementation suffers from very large communication costs, leading to inefficient executions. We thus proceed with the design of a new ROLLUP operator for highlevel languages. Our operator is self-optimizing, which means that it automatically performs load-balancing and determines a suitable operating point to achieve the highest performance. We have implemented our ROLLUP operator for Apache Pig, a popular high-level language in the Hadoop ecosystem. Our experimental results, obtained on both synthetic and real datasets, indicate that our new operator outperforms the current ROLLUP implementation in Pig by at least 50%.

show abstract

Section: A Rollup In Parallel Databasesmentioning

confidence: 99%

Efficient and Self-Balanced ROLLUP Aggregates for Large-Scale Data Summarization

Phan

Hoang-Xuan

Dell’Amico³

et al. 2015

2015 IEEE International Congress on Big Data

View full text Add to dashboard Cite

show abstract

“…Building the data cube can be a massive computational task, and significant research has been published on sequential and parallel data cube construction methods (e.g. [4], [5], [3], [6], [7], [8]). However, the traditional static data cube approach has several disadvantages.…”

Section: A Backgroundmentioning

confidence: 99%

A distributed tree data structure for real-time OLAP on cloud architectures

Dehne

Kong

Rau-Chaplin

et al. 2013

2013 IEEE International Conference on Big Data

View full text Add to dashboard Cite

Abstract-In contrast to queries for on-line transaction processing (OLTP) systems that typically access only a small portion of a database, OLAP queries may need to aggregate large portions of a database which often leads to performance issues. In this paper we introduce CR-OLAP, a Cloud based Real-time OLAP system based on a new distributed index structure for OLAP, the distributed PDCR tree, that utilizes a cloud infrastructure consisting of (m + 1) multi-core processors. With increasing database size, CR-OLAP dynamically increases m to maintain performance. Our distributed PDCR tree data structure supports multiple dimension hierarchies and efficient query processing on the elaborate dimension hierarchies which are so central to OLAP systems. It is particularly efficient for complex OLAP queries that need to aggregate large portions of the data warehouse, such as "report the total sales in all stores located in California and New York during the months February-May of all years". We evaluated CR-OLAP on the Amazon EC2 cloud, using the TPC-DS benchmark data set. The tests demonstrate that CR-OLAP scales well with increasing number of processors, even for complex queries. For example, on an Amazon EC2 cloud instance with eight processors, for a TPC-DS OLAP query stream on a data warehouse with 80 million tuples where every OLAP query aggregates more than 50% of the database, CR-OLAP achieved a query latency of 0.3 seconds which can be considered a real time response.

show abstract

“…Partitioning algorithms to handle iceberg queries with AVERAGE aggregate function have been proposed in [13].Using PC clusters for parallelizing the computation of the iceberg-cube is investigated in [25][26].…”

Section: Bitmap Indicesmentioning

confidence: 99%

Methods for Evaluating Iceberg Queries

Padmapriya¹,

Shanmugapriya²

2013

IJCA

View full text Add to dashboard Cite

Iceberg queries are a special case of SQL queries involving GROUP BY and HAVING clauses, wherein the answer set is small relative to the database size. Iceberg queries have been recently identified as important queries for many applications. Queries can be characterized by their huge input-small output. The iceberg refers to the input, and the tip of it refers to the output. This paper is going to present some of the existing iceberg query processing using data mining.

show abstract

Iceberg-cube computation with PC clusters

Cited by 35 publications

References 9 publications

Efficient and Self-Balanced ROLLUP Aggregates for Large-Scale Data Summarization

Efficient and Self-Balanced ROLLUP Aggregates for Large-Scale Data Summarization

A distributed tree data structure for real-time OLAP on cloud architectures

Methods for Evaluating Iceberg Queries

Contact Info

Product

Resources

About