RTDW-bench: Benchmark for Testing Refreshing Performance of Real-Time Data Warehouse

Jedrzejczak, Jacek; Koszlajda, Tomasz; Wrembel, Robert

doi:10.1007/978-3-642-32597-7_18

Cited by 5 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As SSB, XWeB (Mahboubi & Darmont, 2010) derives from TPC-H, but proposes of a test data warehouse based on a unified reference model for XML warehouses and its associate XQuery decisionsupport workload. RTDW-bench (Jedrzejczak et al, 2012) is also based on TPC-H. It is designed for testing the ability of a real-time data warehouse to handle a transaction stream without delay, given an arrival rate.…”

Section: Decision Support Benchmarksmentioning

confidence: 99%

Database Processing Benchmarks

Darmont

2014

Encyclopedia of Information Science and Technology, Third Edition

View full text Add to dashboard Cite

INTRODUCTIONPerformance measurement tools are very important, both for designers and users of database systems, whether they are aimed at On-Line Transaction Processing (OLTP) or On-Line Analysis Processing (OLAP). Performance evaluation is useful to designers to determine elements of architecture, and more generally to validate or refute hypotheses regarding the actual behavior of a system. Thus, performance evaluation is an essential component in the development process of well-designed and scalable systems, which is nowadays of primary importance in the context of cloud computing. Moreover, users may also employ performance evaluation, either to compare the efficiency of different technologies before selecting a software solution or to tune a system. Performance evaluation by experimentation on a real system is generally referred to as benchmarking. It consists in performing a series of tests on a given system to estimate its performance in a given setting. Typically, a database benchmark is constituted of two main elements: a data model (conceptual schema and extension) and a workload model (set of read and write operations) to apply on this dataset, with respect to a predefined protocol. Most benchmarks also include a set of simple or composite performance metrics such as response time, throughput, number of input/output, disk or memory usage, etc.The aim of this article is to present an overview of the major families of state-of-the-art data processing benchmarks, namely transaction processing benchmarks and decision support benchmarks. We also address the newer trends in cloud benchmarking. Finally, we discuss the issues, tradeoffs and future trends for data processing benchmarks.

show abstract

Section: Decision Support Benchmarksmentioning

confidence: 99%

Database Processing Benchmarks

Darmont

2014

Encyclopedia of Information Science and Technology, Third Edition

View full text Add to dashboard Cite

show abstract

“…Second, real-time data loading and a DW refreshing introduces a problem of benchmarking such an architecture, in particular -generating benchmark data streams as well as defining the set of MVs or MVS to be refreshed. To the best of our knowledge, Jędrzejczak et al (2012) are the first to propose a benchmark for testing a refreshing job of a real-time DW. The benchmark, called RTDW-bench, is based on TPC-H.…”

Section: Benchmarking the Architecturementioning

confidence: 99%

“…Related Work) and therefore there is a need for a standardized benchmark for evaluating the performance of a real-time ELT/ELT and a real-time DW. RTDW-bench (Jędrzejczak et al, 2012) can be further extended and standardized with the content of a data stream, the number of data streams arriving to a RTDW, the number and structure of MVs and their hierarchical dependencies in the MVS; 8. Distributed DW architectures have been proposed for increasing parallel processing of analytical queries, e.g., (Akinde et al, 2003;Furtado, 2009;Yu et al, 2011).…”

Section: Open Research Problemsmentioning

confidence: 99%

On-Demand ELT Architecture for Right-Time BI

Waas¹,

Wrembel

Freudenreich

et al. 2013

International Journal of Data Warehousing and Mining

Self Cite

View full text Add to dashboard Cite

In a typical BI infrastructure, data, extracted from operational data sources, is transformed, cleansed, and loaded into a data warehouse by a periodic ETL process, typically executed on a nightly basis, i.e., a full day’s worth of data is processed and loaded during off-hours. However, it is desirable to have fresher data for business insights at near real-time. To this end, the authors propose to leverage a data warehouse’s capability to directly import raw, unprocessed records and defer the transformation and data cleaning until needed by pending reports. At that time, the database’s own processing mechanisms can be deployed to process the data on-demand. Event-processing capabilities are seamlessly woven into our proposed architecture. Besides outlining an overall architecture, the authors also developed a roadmap for implementing a complete prototype using conventional database technology in the form of hierarchical materialized views.

show abstract

Near real-time with traditional data warehouse architectures

Ferreira

Martins

Furtado

2013

Proceedings of the 17th International Database Engineering &Amp; Applications Symposium on - IDEAS '13

View full text Add to dashboard Cite

Traditional data warehouses integrate new data during lengthy offline periods, with indexes being dropped and rebuilt for efficiency reasons. There is the idea that these and other factors make them unfit for realtime warehousing. We analyze how a set of factors influence near-realtime and frequent loading capabilities, and what can be done to improve near-realtime capacity using a traditional architecture. We analyze how the query workload affects and is affected by the ETL process and the influence of factors such as the type of load strategy, the size of the load data, indexing, integrity constraints, refresh activity over summary data, and fact table partitioning. We evaluate the factors experimentally and show that partitioning is an important factor to deliver near-realtime capacity.

show abstract

RTDW-bench: Benchmark for Testing Refreshing Performance of Real-Time Data Warehouse

Cited by 5 publications

References 17 publications

Database Processing Benchmarks

Database Processing Benchmarks

On-Demand ELT Architecture for Right-Time BI

Near real-time with traditional data warehouse architectures

Contact Info

Product

Resources

About