An analysis and comparison of database cracking kernels

Abstract

Database indexes are a core technique to speed up data retrieval in any kind of data processing system. However, in the presence of schemas with many attributes it becomes infeasible to create indexes for all columns, as maintenance costs and space requirements are simply too high. In these situations, a much more promising approach is to adaptively index the data, i.e. the database gradually partitions (or cracks) those columns that are frequently used in selections. In doing so, the "indexedness" of a table adapts to the requirements of the workload. A large body of work has investigated database cracking, which is a subset of adaptive indexing. Irrespective of their algorithmic behavior, essentially all these works have in common, that the proposed methods use a simple two-sided in-place cracking kernel at the core, which performs a partitioning step. As this partitioning makes a large portion of the total indexing e ort, the choice of the kernel can make a factor of two di erence in the running time for a method sitting on top. To approach the topic, we rst perform an experimental evaluation of existing state-of-the-art kernels and study their respective downsides in detail. Based on the gained insights, we propose both an advanced version of the best existing kernel as well as a new and unconventional approach, which utilizes features of the operating system as well as data parallelism. In our nal evaluation of all kernels, we vary entry size, index layout, selectivity, and number of threads, and provide a decision tree to select the best cracking kernel for the respective situation.

Progressive indexes

“…There is a large body of work on extending and improving database cracking. These improvements include better convergence towards a full index [9,24], more predictable performance [23,12], more efficient tuple reconstruction [15,17,24], better CPU utilization [20], other cracking engines [21,11], predictive query processing [29] and handling updates [14,13]. Below, we give an overview of the work done on improving robustness and general performance.…”

“…Cracking Kernels [21,11] addresses the low CPU efficiency caused by the high number of branch mispredictions. It suggests different cracking kernels that reorganize the elements by exploiting either predication, vectorization, SIMD instructions, or memory rewiring.…”

“…It suggests different cracking kernels that reorganize the elements by exploiting either predication, vectorization, SIMD instructions, or memory rewiring. Haffner et al [11] present a decision tree that recommends the most efficient cracking kernel depending on the query selectivity, type size, and data organization.…”

“…We implemented all our Progressive Indexing algorithms in a stand-alone program written in C++. We included implementations of the adaptive indexing algorithms, provided by the authors, and implemented an adaptive cracking kernel algorithm that picks the most efficient kernel when executing a query, following the decision tree from Haffner et al [11]. Both the progressive indexing algorithms and the existing techniques were compiled with GNU g++ version 7.2.1 using optimization level -O3.…”

“…(2) Adaptive indexing techniques such as database cracking [16,9,24,23,12,15,17,21,20,11,14,13] are a more promising solution. They focus on automatically and incrementally building an index as a side effect of querying the data.…”