The DBO database system

Rusu, Florin; Xu, Fei; Perez, Luis L.; Wu, Mingxi; Jampani, Ravi; Jermaine, Chris; Dobra, Alin

doi:10.1145/1376616.1376740

Cited by 13 publications

(11 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sampling and randomized algorithms have a very long history in databases; the best-known early work is the PhD thesis of Olken [14] and a series of papers from Case Western in the early 1990's [7,8]. Our work on Turbo DBO is a continuation of a line of work on OLA in the database management literature [5,13,6,11,15,9]. The focus in Turbo DBO is on the systems-oriented issues that are important when one designs a database system from the ground-up to utilize randomized algorithms.…”

Section: Does Dbo Always Converge Quickly?mentioning

confidence: 99%

See 1 more Smart Citation

Turbo-charging estimate convergence in DBO

et al. 2009

Self Cite

View full text Add to dashboard Cite

DBO is a database system that utilizes randomized algorithms to give statistically meaningful estimates for the final answer to a multi-table, disk-based query from start to finish during query execution. However, DBO's "time 'til utility" (or "TTU"; that is, the time until DBO can give a useful estimate) can be overly large, particularly in the case that many database tables are joined in a query, or in the case that a join query includes a very selective predicate on one or more of the tables, or when the data are skewed. In this paper, we describe Turbo DBO, which is a prototype database system that can answer multi-table join queries in a scalable fashion, just like DBO. However, Turbo DBO often has a much lower TTU than DBO. The key innovation of Turbo DBO is that it makes use of novel algorithms that look for and remember "partial match" tuples in a randomized fashion. These are tuples that satisfy some of the boolean predicates associated with the query, and can possibly be grown into tuples that actually contribute to the final query result at a later time.

show abstract

Section: Does Dbo Always Converge Quickly?mentioning

confidence: 99%

“…The most recent work on OLA has been undertaken in the context of the DBO database system [15,9]. DBO is unique in that unlike the original OLA proposals-which can only provide an estimate for the final query result as long as at last one table that is being joined fits entirely in memory-DBO is saleable.…”

Section: Introductionmentioning

confidence: 99%

Turbo-charging estimate convergence in DBO

et al. 2009

Self Cite

View full text Add to dashboard Cite

show abstract

“…If φ is a very selective condition, most samples do not contribute to the summation computation of the approximate expectation. (This is closely related to the most prominent problem in online aggregation systems [8], [16], and also in MCDB). A.…”

Section: Sampling and Integrationmentioning

confidence: 92%

PIP: A database system for great and small expectations

Kennedy

2010

2010 IEEE 26th International Conference on Data Engineering (ICDE 2010)

View full text Add to dashboard Cite

Estimation via sampling out of highly selective join queries is well known to be problematic, most notably in online aggregation. Without goal-directed sampling strategies, samples falling outside of the selection constraints lower estimation efficiency at best, and cause inaccurate estimates at worst. This problem appears in general probabilistic database systems, where query processing is tightly coupled with sampling. By committing to a set of samples before evaluating the query, the engine wastes effort on samples that will be discarded, query processing that may need to be repeated, or unnecessarily large numbers of samples. We describe PIP, a general probabilistic database system that uses symbolic representations of probabilistic data to defer computation of expectations, moments, and other statistical measures until the expression to be measured is fully known. This approach is sufficiently general to admit both continuous and discrete distributions. Moreover, deferring sampling enables a broad range of goal-oriented sampling-based (as well as exact) integration techniques for computing expectations, allows the selection of the integration strategy most appropriate to the expression being measured, and can reduce the amount of sampling work required. We demonstrate the effectiveness of this approach by showing that even straightforward algorithms can make use of the added information. These algorithms have a profoundly positive impact on the efficiency and accuracy of expectation computations, particularly in the case of highly selective join queries.

show abstract

“…The database online aggregation literature has its origins in the seminal paper by Hellerstein et al [21]. We can broadly categorize this body of work into system design [30,7,13,2], online join algorithms [20,8,34], and methods to derive confidence bounds [19]. All of this work is targeted at single-node centralized environments.…”

Section: Related Workmentioning

confidence: 99%

Speculative Approximations for Terascale Distributed Gradient Descent Optimization

Qin

Rusu

2015

Proceedings of the Fourth Workshop on Data Analytics in the Cloud

Self Cite

View full text Add to dashboard Cite

Model calibration is a major challenge faced by the plethora of statistical analytics packages that are increasingly used in Big Data applications. Identifying the optimal model parameters is a timeconsuming process that has to be executed from scratch for every dataset/model combination even by experienced data scientists. We argue that the incapacity to evaluate multiple parameter configurations simultaneously and the lack of support to quickly identify sub-optimal configurations are the principal causes.In this paper, we develop two database-inspired techniques for efficient model calibration. Speculative parameter testing applies advanced parallel multi-query processing methods to evaluate several configurations concurrently. Online aggregation is applied to identify sub-optimal configurations early in the processing by incrementally sampling the training dataset and estimating the objective function corresponding to each configuration. We design concurrent online aggregation estimators and define halting conditions to accurately and timely stop the execution.We apply the proposed techniques to distributed gradient descent optimization -batch and incremental -for support vector machines and logistic regression models. We implement the resulting solutions in GLADE PF-OLA -a state-of-the-art Big Data analytics system -and evaluate their performance over terascalesize synthetic and real datasets. The results confirm that as many as 32 configurations can be evaluated concurrently almost as fast as one, while sub-optimal configurations are detected accurately in as little as a 1/20 th fraction of the time.

show abstract

The DBO database system

Cited by 13 publications

References 6 publications

Turbo-charging estimate convergence in DBO

Turbo-charging estimate convergence in DBO

PIP: A database system for great and small expectations

Speculative Approximations for Terascale Distributed Gradient Descent Optimization

Contact Info

Product

Resources

About