UDO: Universal Database Optimization using Reinforcement Learning

Abstract

UDO is a versatile tool for offline tuning of database systems for specific workloads. UDO can consider a variety of tuning choices, reaching from picking transaction code variants over index selections up to database system parameter tuning. UDO uses reinforcement learning to converge to near-optimal configurations, creating and evaluating different configurations via actual query executions (instead of relying on simplifying cost models). To cater to different parameter types, UDO distinguishes heavy parameters (which are expensive to change, e.g. physical design parameters) from light parameters. Specifically for optimizing heavy parameters, UDO uses reinforcement learning algorithms that allow delaying the point at which the reward feedback becomes available. This gives us the freedom to optimize the point in time and the order in which different configurations are created and evaluated (by benchmarking a workload sample). UDO uses a cost-based planner to minimize reconfiguration overheads. For instance, it aims to amortize the creation of expensive data structures by consecutively evaluating configurations using them. We evaluate UDO on Postgres as well as MySQL and on TPC-H as well as TPC-C, optimizing a variety of light and heavy parameters concurrently.

Procrastinated Tree Search: Black-box Optimization with Delayed, Noisy, and Multi-Fidelity Feedback

“…In real-life, the feedbacks of the evaluation oracle can be received after a delay due to the computation time to complete the simulation or evaluation (Weinberger and Ordentlich, 2002), or to complete the communication between servers (Agarwal and Duchi, 2012;Sra et al, 2015). Such delayed feedback is natural in different optimization problems, including the white-box settings (Wang et al, 2021;Li et al, 2019;Joulani et al, 2016;Langford et al, 2009). In some other problems, introducing artificial delays while performing tree search, may create opportunities for work sharing between consecutive evaluations, thereby reducing computation time (Wang et al, 2021).…”

“…Such delayed feedback is natural in different optimization problems, including the white-box settings (Wang et al, 2021;Li et al, 2019;Joulani et al, 2016;Langford et al, 2009). In some other problems, introducing artificial delays while performing tree search, may create opportunities for work sharing between consecutive evaluations, thereby reducing computation time (Wang et al, 2021). This motivated us to look into the delayed feedback for black-box optimization.…”

“…This motivated us to look into the delayed feedback for black-box optimization. Additionally, the feedbacks of the oracle can be noisy or even the objective function itself can be noisy, for example simulation oracles for physical processes (Kajita et al, 2020) and evaluation oracles for hyperparameter tuning of classifiers (Sen et al, 2019) and computer systems (Fischer et al, 2015;Wang et al, 2021). On the other hand, the oracle may invoke a multi-fidelity framework.…”

“…If the expected delay is O(ln T ) , the expected simple regret is practically of the same order as that of non-delayed feedbacks. Thus, in cases of applications where introducing artificial delays helps in improving the computational cost (Wang et al, 2021), we can tune the delays to O(ln T ) for a given horizon T without harming the accuracy. We refer to this range of delays as benign delay.…”

“…Black-box optimization (Munos, 2014;Sen et al, 2019), alternatively known as zeroth-order optimization (Xu et al, 2020) or continuous-arm multi-armed bandit (Bubeck et al, 2011), is a widely studied problem and has been successfully applied in reinforcement learning (Munos, 2014;Grill et al, 2020), neural architecture search (Wang et al, 2019a), large-scale database tuning (Pavlo et al, 2017;Wang et al, 2021), robotics (Martinez-Cantin, 2017), AutoML (Fischer et al, 2015), material science (Xue et al, 2016;Kajita et al, 2020), and many other domains. In black-box optimization, we aim to maximize an unknown function f : X → R, i.e.…”