Iceberg-cube computation with PC clusters

Ng, Raymond T.; Wagner, Alan; Yin, Yu

doi:10.1145/375663.375666

Cited by 48 publications

(15 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Han et al [9] developed a method for computing iceberg queries with non-monotonic aggregate functions. Ng et al [16] investigated iceberg queries on distributed systems. Chen et al [5] explored iceberg cube computation in shared-nothing clusters.…”

Section: Related Workmentioning

confidence: 99%

Multidimensional benchmarking in data warehouses

Campbell

Mao

Pei

et al. 2017

IDA

View full text Add to dashboard Cite

Benchmarking is among the most widely adopted practices in business today. However, to the best of our knowledge, conducting multidimensional benchmarking in data warehouses has not been explored from a technical efficiency perspective. In this paper, we formulate benchmark queries in the context of data warehousing and business intelligence, and develop algorithms to answer benchmark queries efficiently. Our methods employ a few interesting ideas and the state-of-the-art data cube computation techniques to reduce the number of aggregate cells that need to be computed and indexed. An empirical study using the TPC-H and the Weather data sets demonstrates the efficiency and the scalability of our methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Multidimensional benchmarking in data warehouses

Campbell

Mao

Pei

et al. 2017

IDA

View full text Add to dashboard Cite

show abstract

“…Bearing the ideal of making OLAP "truly online", Ng et al [NWY01] develop a collection of parallel algorithms directed towards online and offline creation of data cubes using low cost PC clusters to parallelize computations.…”

Section: Related Workmentioning

confidence: 99%

A linear algebra approach to OLAP

Macedo

Oliveira

2015

Form. Asp. Comput.

View full text Add to dashboard Cite

Inspired by the relational algebra of data processing, this paper addresses the foundations of data analytical processing from a linear algebra perspective. The paper investigates, in particular, how aggregation operations such as cross tabulations and data cubes essential to quantitative analysis of data can be expressed solely in terms of matrix multiplication, transposition and the Khatri-Rao variant of the Kronecker product. The approach offers a basis for deriving an algebraic theory of data consolidation, handling the quantitative as well as qualitative sides of data science in a natural, elegant and typed way. It also shows potential for parallel analytical processing (OLAP), as the parallelization theory of such matrix operations is well acknowledged.

show abstract

“…Building the data cube can be a massive computational task, and significant research has been published on sequential and parallel data cube construction methods (e.g. [7,9,11,12,21,28]). However, the traditional static data cube approach has several disadvantages.…”

Section: Introductionmentioning

confidence: 99%

Parallel Real-Time OLAP on Multi-Core Processors

Dehne

Zaboli

2015

International Journal of Data Warehousing and Mining

View full text Add to dashboard Cite

One of the most powerful and prominent technologies for knowledge discovery in decision support systems is online analytical processing (OLAP). Most of the traditional OLAP research, and most of the commercial systems, follow the static data cube approach proposed by Gray et.al. and materialize all or a subset of the cuboids of the data cube in order to ensure adequate query performance. Practitioners have called for some time for a real-time OLAP approach where the OLAP system gets updated instantaneously as new data arrives and always provides an up-to-date data warehouse for the decision support process. However, a major problem for real-time OLAP is the significant performance issues with large scale data warehouses. The aim of our research is to address these problems through the use of efficient parallel computing methods. In this paper, we present a parallel real-time OLAP system for multi-core processors. To our knowledge, this is the first real-time OLAP system that has been parallelized and optimized for contemporary multi-core architectures. Our system allows for multiple insert and multiple query transactions to be executed in parallel and in real-time. We evaluated our method for a multitude of scenarios (different ratios of insert and query transactions, query transactions with different amounts of data aggregation, different database sizes, etc.), using the TPCDS “Decision Support” benchmark data set. As multi-core test platforms, we used an Intel Sandy Bridge processor with 4 cores (8 hardware supported threads) and an Intel Xeon Westmere processor with 20 cores (40 hardware supported threads). The tests demonstrate that, with increasing number of processor cores, our parallel system achieves close to linear speedup in transaction response time and transaction throughput. On the 20 core architecture we achieved, for a 100 GB database, a better than 0.25 second query response time for real-time OLAP queries that aggregate 25% of the database. Since hardware performance improvements are currently, and in the foreseeable future, achieved not by faster processors but by increasing the number of processor cores, our new parallel real-time OLAP method has the potential to enable OLAP systems that operate in real-time on large databases.

show abstract

Iceberg-cube computation with PC clusters

Cited by 48 publications

References 12 publications

Multidimensional benchmarking in data warehouses

Multidimensional benchmarking in data warehouses

A linear algebra approach to OLAP

Parallel Real-Time OLAP on Multi-Core Processors

Contact Info

Product

Resources

About