DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees

Perera, R. Malinga; Oetomo, Bastian; Rubinstein, Benjamin I. P.; Borovica-Gajić, Renata

doi:10.1109/icde51399.2021.00058

Cited by 25 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PDTool and MCTS on HTAP workloads and summarises results under analytical workloads, reported earlier [40]. We report the total workload time broken down by recommendation, index creation, and workload execution times.…”

Section: This Section Reports Empirical Comparisons Of Mab Againstmentioning

confidence: 96%

“…Therefore, in the remaining experiments, we compare MAB against PDTool (the strongest competitor). Further analysis of additional RL approaches under analytical workloads can be found in [40]. Those experiments demonstrate that deep RL's randomised exploration of the vast state-action space and challenging hyperparameter tuning contributes to the solution volatility, whereas MAB typically provides better convergence and simpler implementation.…”

Section: Static Htap Workloadsmentioning

confidence: 99%

“…PDTool missing the index on Orders.O custkey appears to be more costly with Zipfian factors 2 and 3, mainly affecting Q22. An in-depth analysis of the solution fitness on analytical workloads can be found in [40].…”

Section: Mab Vs Pdtool Under Analytical Workloadsmentioning

confidence: 99%

“…[62] describes RL-based index selection, which depends solely on the recommendation tool for query-level recommendations and is affected by decision combinatorial explosion, both issues addressed in our work. Unlike its more general counterpart (RL), MABs have advantages of faster convergences, simpler implementation, and theoretical guarantees [40]. There has also been recent interest in using bandits for database tasks such as monitoring, query optimisation and join ordering [63], [64], [65].…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

No DBA? No Regret! Multi-Armed Bandits for Index Tuning of Analytical and HTAP Workloads With Provable Guarantees

Perera

Oetomo

Rubinstein

et al. 2023

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

Automating physical database design has remained a long-term interest in database research due to substantial performance gains afforded by optimised structures. Despite significant progress, a majority of today's commercial solutions are highly manual, requiring offline invocation by database administrators (DBAs). This status quo is untenable: identifying representative static workloads is no longer realistic; and physical design tools remain susceptible to the query optimiser's cost misestimates. Furthermore, modern application environments like hybrid transactional and analytical processing (HTAP) systems render analytical modelling next to impossible. We propose a self-driving approach to online index selection that does not depend on the DBA and query optimiser, and instead learns the benefits of viable structures through strategic exploration and direct performance observation. We view the problem as one of sequential decision making under uncertainty, specifically within the bandit learning setting. Multi-armed bandits balance exploration and exploitation to provably guarantee average performance that converges to policies that are optimal with perfect hindsight. Our comprehensive empirical evaluation against a state-of-the-art commercial tuning tool demonstrates up to 75% speed-up in analytical processing environments and 59% speed-up in HTAP environments. Lastly, our bandit framework outperforms a Monte Carlo tree search (MCTS)-based database optimiser, providing up to 24% speed-up.

show abstract

Section: This Section Reports Empirical Comparisons Of Mab Againstmentioning

confidence: 96%

Section: Static Htap Workloadsmentioning

confidence: 99%

Section: Mab Vs Pdtool Under Analytical Workloadsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

No DBA? No Regret! Multi-Armed Bandits for Index Tuning of Analytical and HTAP Workloads With Provable Guarantees

Perera

Oetomo

Rubinstein

et al. 2023

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Contextual BO considers the environmental conditions by augmenting the GP kernel with extra context variables and uses 𝐶𝐺𝑃 − 𝑈𝐶𝐵 to select promising action [35]. DBA Bandit [52] chooses a set of indices from finite and discrete configuration space based on the context of indexed columns and derived statistics from database optimizer. It achieves an Õ ( √ 𝑛) regret bound after playing 𝑛 rounds as a safety guarantee, implying that the per-step average cumulative regret approaches zero after sufficiently many steps.…”

Section: Related Workmentioning

confidence: 99%

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Zhang,

Wu,

et al. 2022

Preprint

View full text Add to dashboard Cite

Configuration knobs of database systems are essential to achieve high throughput and low latency. Recently, automatic tuning systems using machine learning methods (ML) have shown to find better configurations compared to experienced database administrators (DBAs). However, there are still gaps to apply the existing systems in production environments, especially in the cloud. First, they conduct tuning for a given workload within a limited time window and ignore the dynamicity of workloads and data. Second, they rely on a copied instance and do not consider the availability of the database when sampling configurations, making the tuning expensive, delayed, and unsafe. To fill these gaps, we propose OnlineTune, which tunes the online databases safely in changing cloud environments. To accommodate the dynamicity, OnlineTune embeds the environmental factors as context feature and adopts contextual Bayesian Optimization with context space partition to optimize the database adaptively and scalably. To pursue safety during tuning, we leverage the black-box and the white-box knowledge to evaluate the safety of configurations and propose a safe exploration strategy via subspace adaptation.We conduct evaluations on dynamic workloads from benchmarks and real-world workloads. Compared with the state-of-the-art methods, OnlineTune achieves 14.4%~165.3% improvement on cumulative performance while reducing 91.0%~99.5% unsafe configuration recommendations. CCS CONCEPTS• Information systems → Autonomous database administration; • Computing methodologies → Machine learning.

show abstract

A Divergent Index Advisor Using Deep Reinforcement Learning

Sadri

Gruenwald

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees

Cited by 25 publications

References 32 publications

No DBA? No Regret! Multi-Armed Bandits for Index Tuning of Analytical and HTAP Workloads With Provable Guarantees

No DBA? No Regret! Multi-Armed Bandits for Index Tuning of Analytical and HTAP Workloads With Provable Guarantees

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

A Divergent Index Advisor Using Deep Reinforcement Learning

Contact Info

Product

Resources

About