Row-wise parallel predicate evaluation

Zhou

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

et al. 2016

Abstract-Denormalization is a common tactic for enhancing performance of data warehouses, though its side-effect is quite obvious. Besides being confronted with update abnormality, denormalization has to consume additional storage space. As a result, this tactic is rarely used in main memory databases, which regards storage space, i.e., RAM, as scarce resource. Nevertheless, our research reveals that main memory database can benefit enormously from denormalization, as it is able to remarkably simplify the query processing plans and reduce the computation cost. In this paper, we present A-Store, a main memory OLAP engine customized for star/snowflake schemas. Instead of generating fully materialized denormalization, A-Store resorts to virtual denormalization by treating array indexes as primary keys. This design allows us to harvest the benefit of denormalization without sacrificing additional RAM space. A-Store uses a generic query processing model for all SPJGA queries. It applies a number of state-of-the-art optimization methods, such as vectorized scan and aggregation, to achieve superior performance. Our experiments show that A-Store outperforms the most prestigious MMDB systems significantly in star/snowflake schema based query processing.

Section: Related Workmentioning

confidence: 99%

Virtual denormalization via array index reference for main memory OLAP

Zhou

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

et al. 2016

2013 IEEE 29th International Conference on Data Engineering (ICDE)

“…To increase the CPU efficiency of MRDBMS, the database research community has proposed a number of techniques [5], [35], [20], [17]. The most prominent ones are bulk processing and a-priory query compilation .…”

Section: A Cpu Efficient Processingmentioning

confidence: 99%

“…Due to the inflexibility of the primitives, however, bulk processing is only efficient on fully decomposed relations, which are known to yield poor cache locality for OLTP applications. Using tuple clustering, compression and bank packing [20], it is possible to efficiently evaluate selections on multiple attributes in a bulk-manner. However the necessary compression hurts update performance and decompression adds to tuple reconstruction costs.…”

Section: A Cpu Efficient Processingmentioning

confidence: 99%

CPU and cache efficient management of memory-resident databases

Pirk

Funke²,

Grund

et al. 2013

Abstract-Memory-Resident Database Management Systems (MRDBMS) have to be optimized for two resources: CPU cycles and memory bandwidth. To optimize for bandwidth in mixed OLTP/OLAP scenarios, the hybrid or Partially Decomposed Storage Model (PDSM) has been proposed. However, in current implementations, bandwidth savings achieved by partial decomposition come at increased CPU costs. To achieve the aspired bandwidth savings without sacrificing CPU efficiency, we combine partially decomposed storage with Just-in-Time (JiT) compilation of queries, thus eliminating CPU inefficient function calls. Since existing cost based optimization components are not designed for JiT-compiled query execution, we also develop a novel approach to cost modeling and subsequent storage layout optimization. Our evaluation shows that the JiT-based processor maintains the bandwidth savings of previously presented hybrid query processors but outperforms them by two orders of magnitude due to increased CPU efficiency.

“…In the above-mentioned example, a predicate such as State = "California" can be applied on the encoded value as State = 000101. Moreover, the compact encoding of the values permits loading multiple values for a column into a register, so that a predicate can be applied simultaneously to all those values in a vector comparison [9,12,18,22].…”

Section: Introductionmentioning

confidence: 99%

Joins on encoded and partitioned data

et al. 2014

Self Cite

Compression has historically been used to reduce the cost of storage, I/Os from that storage, and buffer pool utilization, at the expense of the CPU required to decompress data every time it is queried. However, significant additional CPU efficiencies can be achieved by deferring decompression as late in query processing as possible and performing query processing operations directly on the still-compressed data. In this paper, we investigate the benefits and challenges of performing joins on compressed (or encoded) data. We demonstrate the benefit of independently optimizing the compression scheme of each join column, even though join predicates relating values from multiple columns may require translation of the encoding of one join column into the encoding of the other. We also show the benefit of compressing "payload" data other than the join columns "on the fly," to minimize the size of hash tables used in the join. By partitioning the domain of each column and defining separate dictionaries for each partition, we can achieve even better overall compression as well as increased flexibility in dealing with new values introduced by updates. Instead of decompressing both join columns participating in a join to resolve their different compression schemes, our system performs a light-weight mapping of only qualifying rows from one of the join columns to the encoding space of the other at run time. Consequently, join predicates can be applied directly on the compressed data. We call this procedure encoding translation. Two alternatives of encoding translation are developed and compared in the paper. We provide a comprehensive evaluation of these alternatives using product implementations of each on the TPC-H data set, and demonstrate that performing joins on encoded and partitioned data achieves both superior performance and excellent compression.