ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases

Zhang, Xinyi; Wu, Huanmei; Chang, Zhuo; Jin, Shuowei; Tan, Jian; Li, Feifei; Zhang, Tieying; Cui, Bin

doi:10.1145/3448016.3457291

Cited by 68 publications

(31 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It embeds workload features to predict internal metrics via a pre-trained model, while CDBTune uses the measured internal metrics. • ResTune adopts constrained Bayesian Optimization to minimize resource utilization with SLA constraints [74]. It uses an ensemble framework (i.e., RGPE) to transfer historical knowledge from observations of source workloads.…”

Section: Discussionmentioning

confidence: 99%

“…• Learning-based. iTuned [15], Ottertune [6] and ResTune [74] use Bayesian Optimization (BO) based method, modeling the tuning as a black-box optimization problem. Reinforcement Learning (RL) is adopted in [41,73] to tune DBMS by learning a neural network between the internal metrics and the configurations.…”

Section: Related Workmentioning

confidence: 99%

“…SQL text parsing directly processes the query texts. TF-IDF is used to represent a query as a collection of the weighted frequencies of its individual word tokens [45,74]. However, its unbounded vocabulary makes generalization across workloads difficult.…”

Section: Related Workmentioning

confidence: 99%

“…However, its unbounded vocabulary makes generalization across workloads difficult. Therefore ResTune restricts its calculation to reserved SQL keywords [74]. To support larger vocabulary, [30,76] resort to representation learning which is frequently used in NLP.…”

Section: Related Workmentioning

confidence: 99%

“…In case the default performance fluctuates with workload changes, we assume that the default performance for any given workload can be acquired. Note in practice, the default performance can be easily obtained or predicted when a historical knowledge base is available (such as in [6,18,74]). For example, the user could train a regression model that inputs the context and outputs the default performance.…”

Section: Problem Statementmentioning

confidence: 99%

See 4 more Smart Citations

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Zhang,

Wu,

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Configuration knobs of database systems are essential to achieve high throughput and low latency. Recently, automatic tuning systems using machine learning methods (ML) have shown to find better configurations compared to experienced database administrators (DBAs). However, there are still gaps to apply the existing systems in production environments, especially in the cloud. First, they conduct tuning for a given workload within a limited time window and ignore the dynamicity of workloads and data. Second, they rely on a copied instance and do not consider the availability of the database when sampling configurations, making the tuning expensive, delayed, and unsafe. To fill these gaps, we propose OnlineTune, which tunes the online databases safely in changing cloud environments. To accommodate the dynamicity, OnlineTune embeds the environmental factors as context feature and adopts contextual Bayesian Optimization with context space partition to optimize the database adaptively and scalably. To pursue safety during tuning, we leverage the black-box and the white-box knowledge to evaluate the safety of configurations and propose a safe exploration strategy via subspace adaptation.We conduct evaluations on dynamic workloads from benchmarks and real-world workloads. Compared with the state-of-the-art methods, OnlineTune achieves 14.4%~165.3% improvement on cumulative performance while reducing 91.0%~99.5% unsafe configuration recommendations. CCS CONCEPTS• Information systems → Autonomous database administration; • Computing methodologies → Machine learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Problem Statementmentioning

confidence: 99%

See 3 more Smart Citations

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Zhang,

Wu,

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

JointConf: Jointly autotuning configuration parameters for modularized graph databases

Dou

Mei

Zhang

et al. 2022

J Software Evolu Process

View full text Add to dashboard Cite

To support different application scenarios, graph databases (GDBs) usually provide a large number of performance-related parameters for developers. Since manually configuring is both time-consuming and cost-intensive, automatically tuning configurations parameters to achieve a better performance has been an urgent need. Besides, considering various graph management requirements, GDBs begin to utilize the modular architecture to interoperate with a wide range of storage and index backends. Due to the complicated interactions among different modules, sequentially tuning each software with previous solutions may fall into a local optimal and it is necessary to jointly autotune the cross-module configuration parameters. Toward this challenging target, we propose JointConf-a new black-box approach of jointly autotuning configuration parameters for modularized GDBs. To address the formulated highdimensional black-box optimization problem, JointConf utilizes the recently proposed BO_dropout algorithm. Inspired by the dropout algorithm in neural networks, BO_dropout explores efficient dimension dropout to achieve a high-dimensional Bayesian optimization. We evaluate the effectiveness of JointConf on a local distributed JanusGraph cluster with three different graph query benchmark applications and experimental results show its advantages over the four baseline search-based approaches. The necessity of jointly tuning for modularized GDBs is also verified in our experiments.

show abstract