Cleaning Denial Constraint Violations through Relaxation

Giannakopoulou, Stella; Karpathiotakis, Manos; Ailamaki, Anastasia

doi:10.1145/3318464.3389775

Cited by 28 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, CleanDB addresses a different dimension of the scalability issue of data cleaning than QueryER does. Daisy [14] is a system that performs probabilistic repair of functional dependency violations with query-result relaxation that enables interleaving SPJ queries. Daisy also introduces update operators which differentiate between Select and Join operators, inside the query plan by analyzing the query operators affected by the constraints.…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Analysis-aware data processing refers to an exploratory analysis scenario, where users apply traditional data integration methods, such as cleaning [14,31], during query time. Several approaches extend the capabilities of SQL engines with operators that relax the results of the query, by repairing inconsistent data [14]. Analysisaware Entity Resolution (ER) is a special case which aims at extending the results of the query by resolving duplicate entities (records that represent the same real-world entity) during query time [1][2][3].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

QueryER: A Framework for Fast Analysis-Aware Deduplication over Dirty Data

Alexiou¹,

Papastefanatos²,

Stamatopoulos³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this work, we explore the problem of correctly and efficiently answering complex SPJ queries issued directly on top of dirty data. We introduce QueryER, a framework that seamlessly integrates Entity Resolution into Query Processing. QueryER executes analysisaware deduplication by weaving ER operators into the query plan. The experimental evaluation of our approach exhibits that it adapts to the workload and scales on both real and synthetic datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

QueryER: A Framework for Fast Analysis-Aware Deduplication over Dirty Data

Alexiou¹,

Papastefanatos²,

Stamatopoulos³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Active learning approaches [33,54,59] ask users to verify whether repair candidates are correct. Daisy [18] uses categorical histograms to identify and repair errors in join attributes. In Reptile, the user submits a single complaint over an aggregate query result, and the system trains multi-level models.…”

Section: Related Workmentioning

confidence: 99%

Reptile: Aggregation-level Explanations for Hierarchical Data

Huang¹,

Wu²

2021

Preprint

View full text Add to dashboard Cite

Recent query explanation systems help users understand anomalies in aggregation results by proposing predicates that describe input records that, if deleted, would resolve the anomalies. However, it can be difficult for users to understand how a predicate was chosen, and these approaches are limited to errors that can be resolved through deletion. In contrast, data errors may be due to group-wise errors, such as missing records or systematic value errors.This paper presents Reptile, an explanation system for hierarchical data. Given an anomalous aggregate query result, Reptile recommends the next drill-down attribute, and ranks the drill-down groups based on the extent repairing the group's statistics to its expected values resolves the anomaly. Reptile efficiently trains a multi-level model that leverages the data's hierarchy to estimate the expected values, and uses a factorised representation of the feature matrix to remove redundancies due to the data's hierarchical structure. We further extend model training to support factorised data, and develop a suite of optimizations that leverage the data's hierarchical structure. Reptile reduces end-to-end runtimes by >6× compared to a Matlab-based implementation, correctly identifies 21/30 data errors in John Hopkin's COVID-19 data, and correctly resolves 20/22 complaints in a user study using data and researchers from Columbia University's Financial Instruments Sector Team.

show abstract

“…Finally, it should be noted that there is a large body of work on managing inconsistent databases via data cleaning. Two recent systems in this area are HoloClean [37] and Daisy [22,23]. There are fundamental differences between data cleaning systems and systems for consistent query answering.…”

Section: Introductionmentioning

confidence: 99%

Consistent Answers of Aggregation Queries using SAT Solvers

Dixit¹,

Kolaitis²

2021

Preprint

View full text Add to dashboard Cite

The framework of database repairs and consistent answers to queries is a principled approach to managing inconsistent databases. We describe the first system able to compute the consistent answers of general aggregation queries with the COUNT(𝐴), COUNT(*), and SUM operators, and with or without grouping constructs. Our system uses reductions to optimization versions of Boolean satisfiability (SAT) and then leverages powerful SAT solvers. We carry out an extensive set of experiments on both synthetic and real-world data that demonstrate the usefulness and scalability of this approach.

show abstract

Cleaning Denial Constraint Violations through Relaxation

Cited by 28 publications

References 19 publications

QueryER: A Framework for Fast Analysis-Aware Deduplication over Dirty Data

QueryER: A Framework for Fast Analysis-Aware Deduplication over Dirty Data

Reptile: Aggregation-level Explanations for Hierarchical Data

Consistent Answers of Aggregation Queries using SAT Solvers

Contact Info

Product

Resources

About