PatchIndex - Exploiting Approximate Constraints in Self-managing Databases

Kläbe, Steffen; Sattler, Kai-Uwe; Baumann, S.

doi:10.1109/icdew49219.2020.00014

Cited by 4 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When used for query optimization, approximate dependencies can cause incomplete and incorrect results. Hence, they are in general not usable for query optimization unless they are implemented in approximate query processing systems [90], other data structures compensate their optimization mistakes [70], or they are used only for cardinality and cost estimation optimizations (see Table 1). Conditions Data dependencies are sometimes tied to conditions that limit their scope.…”

Section: Data Dependency Propertiesmentioning

confidence: 99%

Data dependencies for query optimization: a survey

2021

View full text Add to dashboard Cite

Effective query optimization is a core feature of any database management system. While most query optimization techniques make use of simple metadata, such as cardinalities and other basic statistics, other optimization techniques are based on more advanced metadata including data dependencies, such as functional, uniqueness, order, or inclusion dependencies. This survey provides an overview, intuitive descriptions, and classifications of query optimization and execution strategies that are enabled by data dependencies. We consider the most popular types of data dependencies and focus on optimization strategies that target the optimization of relational database queries. The survey supports database vendors to identify optimization opportunities as well as DBMS researchers to find related work and open research questions.

show abstract

Section: Data Dependency Propertiesmentioning

confidence: 99%

Data dependencies for query optimization: a survey

2021

View full text Add to dashboard Cite

show abstract

“…infering missing values according to an observed data distribution or deleting tuples that violate a constraint. For cases where data manipulation is not desired, we introduced the concept of PatchIndexes in [12] as an alternative approach, enabling database systems to define approximate constraints without modifying data. These constraints hold for all values of an indexed column except a set of exceptions maintained by the PatchIndex.…”

Section: Introductionmentioning

confidence: 99%

“…Cloud warehouse solutions are typically designed as Massively Parallel Processing (MPP) systems, where the key for efficient query processing is to partition and distribute data within the system in order to query it in a distributed fashion. With this paper, we apply and extend the concept of PatchIndexes presented in [12] to parallel and distributed environments. Our main contributions are:…”

Section: Introductionmentioning

confidence: 99%

PatchIndex: exploiting approximate constraints in distributed databases

Kläbe

Sattler

Baumann

2021

Distrib Parallel Databases

Self Cite

View full text Add to dashboard Cite

Cloud data warehouse systems lower the barrier to access data analytics. These applications often lack a database administrator and integrate data from various sources, potentially leading to data not satisfying strict constraints. Automatic schema optimization in self-managing databases is difficult in these environments without prior data cleaning steps. In this paper, we focus on constraint discovery as a subtask of schema optimization. Perfect constraints might not exist in these unclean datasets due to a small set of values violating the constraints. Therefore, we introduce the concept of a generic PatchIndex structure, which handles exceptions to given constraints and enables database systems to define these approximate constraints. We apply the concept to the environment of distributed databases, providing parallel index creation approaches and optimization techniques for parallel queries using PatchIndexes. Furthermore, we describe heuristics for automatic discovery of PatchIndex candidate columns and prove the performance benefit of using PatchIndexes in our evaluation.

show abstract

“…These exceptions prohibit the definition of a constraint, resulting in the loss of valuable information. Examples for approximate constraints are "nearly unique columns" (NUC) and "nearly sorted columns" (NSC), whose definition we introduced in [18].…”

Section: Introductionmentioning

confidence: 99%

“…In [18] we proposed the concept of PatchIndexes, which allows the definition of approximate constraints and enables database systems to benefit from the constraint information in query execution by materializing exceptions and handling them separately. We provided constraint discovery mechanisms for NUC and NSC and compared different design approaches.…”

Section: Introductionmentioning

confidence: 99%

Updatable Materialization of Approximate Constraints

Kläbe¹,

Sattler²,

Baumann³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Modern big data applications integrate data from various sources. As a result, these datasets may not satisfy perfect constraints, leading to sparse schema information and non-optimal query performance. The existing approach of PatchIndexes enable the definition of approximate constraints and improve query performance by exploiting the materialized constraint information. As real world data warehouse workloads are often not limited to read-only queries, we enhance the PatchIndex structure towards an update-conscious design in this paper. Therefore, we present a sharded bitmap as the underlying data structure which offers efficient update operations, and describe approaches to maintain approximate constraints under updates, avoiding index recomputations and full table scans. In our evaluation, we prove that PatchIndexes significantly impact query performance while achieving lightweight update support.

show abstract

PatchIndex - Exploiting Approximate Constraints in Self-managing Databases

Cited by 4 publications

References 9 publications

Data dependencies for query optimization: a survey

Data dependencies for query optimization: a survey

PatchIndex: exploiting approximate constraints in distributed databases

Updatable Materialization of Approximate Constraints

Contact Info

Product

Resources

About