Estimating the Impact of Unknown Unknowns on Aggregate Query Results

Chung, Yeounoh; Mortensen, Michael Lind; Binnig, Carsten; Kraska, Tim

doi:10.1145/3167970

Cited by 10 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The techniques for leveraging aggregate knowledge such as iterative proportional fitting could seamlessly be integrated in our approach. Chung et al [9] estimate the impact of missing tuples on aggregate queries when several data sources are integrated by observing reoccurring tuples. While ReStore similarly helps for the case of different data sources with varying quality again only the single table case is discussed here.…”

Section: Related Workmentioning

confidence: 99%

ReStore - Neural Data Completion for Relational Databases

Hilprecht

Binnig

2021

Proceedings of the 2021 International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Classical approaches for OLAP assume that the data of all tables is complete. However, in case of incomplete tables with missing tuples, classical approaches fail since the result of a SQL aggregate query might significantly differ from the results computed on the full dataset. Today, the only way to deal with missing data is to manually complete the dataset which causes not only high efforts but also requires good statistical skills to determine when a dataset is actually complete. In this paper, we propose an automated approach for relational data completion called ReStore 1 using a new class of (neural) schema-structured completion models that are able to synthesize data which resembles the missing tuples. As we show in our evaluation, this efficiently helps to reduce the relative error of aggregate queries by up to 390% on real-world data compared to using the incomplete data directly for query answering.

show abstract

Section: Related Workmentioning

confidence: 99%

ReStore - Neural Data Completion for Relational Databases

Hilprecht

Binnig

2021

Proceedings of the 2021 International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

show abstract

“…We therefore started to develop techniques that estimate not only the amount of missing data based on techniques from [100,101,102] but also the impact those items might have on query results [19,20]. We assume a simple data integration scenario in which (semi-)independent data sources are integrated into a single database.…”

Section: Uncertainty As Unknown Unknownsmentioning

confidence: 99%

“…The overlap between the different data sets allows us to estimate the number of missing items using species estimation techniques [100]. Further, it is possible to make estimates about the values the missing items might have using our novel bucket estimator [19]. This way, Vizdom is able to indicate to the user how much impact missing data might have on the visualization.…”

Section: Uncertainty As Unknown Unknownsmentioning

confidence: 99%

Northstar

Kraska

2018

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

In order to democratize data science, we need to fundamentally rethink the current analytics stack, from the user interface to the "guts." Most importantly, enabling a broader range of users to unfold the potential of (their) data requires a change in the interface and the "protection" we offer them. On the one hand, visual interfaces for data science have to be intuitive, easy, and interactive to reach users without a strong background in computer science or statistics. On the other hand, we need to protect users from making false discoveries. Furthermore, it requires that technically involved (and often boring) tasks have to be automatically done by the system so that the user can focus on contributing their domain expertise to the problem. In this paper, we present Northstar, the Interactive Data Science System, which we have developed over the last 4 years to explore designs that make advanced analytics and model building more accessible.

show abstract

“…Unfortunately, existing species estimation techniques [30,10] to estimate the completeness of a set, do not consider that workers can make mistakes. At the same time in any real data cleaning scenario, workers can make both false negative errors (a worker fails to identify a true error) and false positive errors (a worker misclassifies a clean item as dirty).…”

Section: Our Goal and Approachmentioning

confidence: 99%

“…To the best of our knowledge, this is the first work to consider species estimation for data quality quantification. Species estimation has been studied in prior work for distinct count estimation and crowdsourcing [18,29,10]. However, the previous work only considered species estimation on clean data without false positive and false negative errors, which is inherent to the data quality estimation setting.…”

Section: Related Workmentioning

confidence: 99%

A Data Quality Metric (DQM): How to Estimate The Number of Undetected Errors in Data Sets

Chung¹,

Krishnan²,

Kraska³

2016

Preprint

Self Cite

View full text Add to dashboard Cite

Data cleaning, whether manual or algorithmic, is rarely perfect leaving a dataset with an unknown number of false positives and false negatives after cleaning. In many scenarios, quantifying the number of remaining errors is challenging because our data integrity rules themselves may be incomplete, or the available gold-standard datasets may be too small to extrapolate. As the use of inherently fallible crowds becomes more prevalent in data cleaning problems, it is important to have estimators to quantify the extent of such errors. We propose novel species estimators to estimate the number of distinct remaining errors in a dataset after it has been cleaned by a set of crowd workers -essentially, quantifying the utility of hiring additional workers to clean the dataset. This problem requires new estimators that are robust to false positives and false negatives, and we empirically show on three real-world datasets that existing species estimators are unstable for this problem, while our proposed techniques quickly converge.

show abstract

Estimating the Impact of Unknown Unknowns on Aggregate Query Results

Cited by 10 publications

References 36 publications

ReStore - Neural Data Completion for Relational Databases

ReStore - Neural Data Completion for Relational Databases

Northstar

A Data Quality Metric (DQM): How to Estimate The Number of Undetected Errors in Data Sets

Contact Info

Product

Resources

About