GSLPI: A Cost-Based Query Progress Indicator

Li, Jiexing; Nehme, Rimma V.; Naughton, Jeffrey F.

doi:10.1109/icde.2012.74

Cited by 22 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, on the left of the display above the tree synopsis, is shown a query progress indicator. It estimates the completion percentage of the running query using techniques like those in [4].…”

Section: Tree Viewsmentioning

confidence: 99%

Vqa

Simitsis

Wilkinson

Blais³

et al. 2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

Database query monitoring tools collect performance metrics, such as memory and cpu usage, while a query is executing and make them available through log files or system tables. The metrics can be used to understand and diagnose query performance issues. However, analytic queries over big data presents new challenges for query monitoring tools. A long-running query may generate tens of thousands of values so simply reporting the metrics may overwhelm the user. Second, analytic queries may be written by database novices who have trouble interpreting the metrics. Third, analytic queries may access data or processing outside the database through user-defined functions and connectors. The impact of these on query performance must be understood. Vertica Query Analyzer (VQA) is a query monitoring tool to address these challenges. VQA is both a useful tool and a research platform for query analytics. It presents query performance metrics through a variety of views and granularities. In addition, it analyzes the metrics for typical performance problems and suggests corrective actions. We demonstrate VQA using TPC-DS queries which have a wide range of query duration and complexity.

show abstract

Section: Tree Viewsmentioning

confidence: 99%

Vqa

Simitsis

Wilkinson

Blais³

et al. 2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…There has been a lot of work in the area of query execution time estimation [5,6,16,18,23]. Unlike previous work, we do not need to get perfect time estimates to make a good data partitioning recommendation.…”

Section: Quantifying Performance Differencesmentioning

confidence: 99%

“…We utilize the optimizer estimated cost units to define the work for an operator in a pipeline. We follow the idea presented in [16] to calculate the cost for a pipeline, and the interested reader is referred to that paper for details.…”

Section: Estimating the Cost Of A Pipelinementioning

confidence: 99%

See 1 more Smart Citation

Resource bricolage for parallel database systems

Naughton

Nehme

2014

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

Running parallel database systems in an environment with heterogeneous resources has become increasingly common, due to cluster evolution and increasing interest in moving applications into public clouds. For database systems running in a heterogeneous cluster, the default uniform data partitioning strategy may overload some of the slow machines while at the same time it may under-utilize the more powerful machines. Since the processing time of a parallel query is determined by the slowest machine, such an allocation strategy may result in a significant query performance degradation.We take a first step to address this problem by introducing a technique we call resource bricolage that improves database performance in heterogeneous environments. Our approach quantifies the performance differences among machines with various resources as they process workloads with diverse resource requirements. We formalize the problem of minimizing workload execution time and view it as an optimization problem, and then we employ linear programming to obtain a recommended data partitioning scheme. We verify the effectiveness of our technique with an extensive experimental study on a commercial database system.

show abstract

“…Another related research direction is query progress indicators [7], [18], [19], [21]. The task of a progress indicator is to dynamically monitor the percentage of work that has been done so far for the query.…”

Section: Related Workmentioning

confidence: 99%

Predicting query execution time: Are optimizer cost models really unusable?

Chen

Zhu

et al. 2013

2013 IEEE 29th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Abstract-Predicting query execution time is useful in many database management issues including admission control, query scheduling, progress monitoring, and system sizing. Recently the research community has been exploring the use of statistical machine learning approaches to build predictive models for this task. An implicit assumption behind this work is that the cost models used by query optimizers are insufficient for query execution time prediction. In this paper we challenge this assumption and show while the simple approach of scaling the optimizer's estimated cost indeed fails, a properly calibrated optimizer cost model is surprisingly effective. However, even a well-tuned optimizer cost model will fail in the presence of errors in cardinality estimates. Accordingly we investigate the novel idea of spending extra resources to refine estimates for the query plan after it has been chosen by the optimizer but before execution. In our experiments we find that a well calibrated query optimizer model along with cardinality estimation refinement provides a low overhead way to provide estimates that are always competitive and often much better than the best reported numbers from the machine learning approaches.

show abstract

GSLPI: A Cost-Based Query Progress Indicator

Cited by 22 publications

References 15 publications

Vqa

Vqa

Resource bricolage for parallel database systems

Predicting query execution time: Are optimizer cost models really unusable?

Contact Info

Product

Resources

About