Scheduling OLTP transactions via learned abort prediction

Abstract

Current main memory database system architectures are still challenged by high contention workloads and this challenge will continue to grow as the number of cores in processors continues to increase [35]. These systems schedule transactions randomly across cores to maximize concurrency and to produce a uniform load across cores. Scheduling never considers potential conflicts. Performance could be improved if scheduling balanced between concurrency to maximize throughput and scheduling transactions linearly to avoid conflicts. In this paper, we present the design of several intelligent transaction scheduling algorithms that consider both potential transaction conflicts and concurrency.To incorporate reasoning about transaction conflicts, we develop a supervised machine learning model that estimates the probability of conflict. This model is incorporated into several scheduling algorithms. In addition, we integrate an unsupervised machine learning algorithm into an intelligent scheduling algorithm. We then empirically measure the performance impact of different scheduling algorithms on OLTP and social networking workloads. Our results show that, with appropriate settings, intelligent scheduling can increase throughput by 54% and reduce abort rate by 80% on a 20-core machine, relative to random scheduling. In summary, the paper provides preliminary evidence that intelligent scheduling significantly improves DBMS performance.Transaction aborts are one of the main sources of performance loss in main memory OLTP systems [35].Current architectures for OLTP DBMS use random scheduling to assign transactions to threads. Random scheduling achieves uniform load across CPU cores and keeps all cores occupied. For workloads with a high abort rate, a large portion of work done by CPU is wasted. In contrast, the abort rate drops to zero if all transactions are scheduled sequentially into one thread. No work is wasted through aborts, but the throughput drops to the performance of a single hardware thread. Research has shown that statistical scheduling of transactions using a history can achieve low abort rate and high throughput [37] for partitionable workloads. We propose a more systematic machine learning approach to schedule transactions that achieves low abort rate and high throughput for both partitionable and non-partitionable workloads. 2 Y. Sheng, et al.The fundamental intuitions of this paper are that (i) the probability that a transactions will abort will high probability can be modeled through machine learning, and (ii) given that an abort is predicted, scheduling can avoid the conflict without loss of response time or throughput. Several research works [1,20,25,27] perform exact analyses of aborts of transaction statements that are complex and not easily generalizable over different workloads or DBMS architectures. Our approach uses machine learning algorithms to model the probability of aborts. Our approach is to (i) build a machine learning model that helps to group transactions that are likely to abort with each ot...

Automatic configuration of the Cassandra database using irace

“…This includes using machine learning algorithms to learn indexes and data structures ( Kraska et al, 2018 ; Wu et al, 2019 ). Another important application is learning the patterns of transactions, in order to predict future operations and schedule them more efficiently ( Ma et al, 2018 ; Sheng et al, 2019 ).…”

Self-Tuning Query Scheduling for Analytical Workloads

“…Here, query performance often deteriorates due to excessive aborts and high lock contention. Performance can mainly be improved by restricting which queries are executed concurrently [21,24]. Similar to our work, both [10] and [24] implement self-tuning capabilities in their scheduler.…”

“…Performance can mainly be improved by restricting which queries are executed concurrently [21,24]. Similar to our work, both [10] and [24] implement self-tuning capabilities in their scheduler. Nevertheless, the utilization of observed workload characteristics in order to improve scheduling decisions is not a recent idea [17,25].…”

Robustness against Read Committed for Transaction Templates

“…Many approaches to increase transaction throughput have been proposed: improved or novel pessimistic (cf., e.g., [28,39,41,47,51]) or optimistic (cf., e.g., [11, 12, 17, 18, 26, 27, 29, 31-33, 36, 42, 43, 52, 53]) algorithms, as well as approaches based on coordination avoidance (cf., e.g., [19,20,35,38,40,45,46]). We do not compare to these as our focus lies on a technique that can be applied to standard DBMS's without any modifications to the database internals.…”