Design Principles for Scaling Multi-core OLTP Under High Contention

Abstract

Although significant recent progress has been made in improving the multi-core scalability of high throughput transactional database systems, modern systems still fail to achieve scalable throughput for workloads involving frequent access to highly contended data. Most of this inability to achieve high throughput is explained by the fundamental constraints involved in guaranteeing ACID -the addition of cores results in more concurrent transactions accessing the same contended data for which access must be serialized in order to guarantee isolation. Thus, linear scalability for contended workloads is impossible. However, there exist flaws in many modern architectures that exacerbate their poor scalability, and result in throughput that is much worse than fundamentally required by the workload.In this paper we identify two prevalent design principles that limit the multi-core scalability of many (but not all) transactional database systems on contended workloads: the multi-purpose nature of execution threads in these systems, and the lack of advanced planning of data access. We demonstrate the deleterious results of these design principles by implementing a prototype system, OR-THRUS, that is motivated by the principles of separation of database component functionality and advanced planning of transactions. We find that these two principles alone result in significantly improved scalability on high-contention workloads, and an order of magnitude increase in throughput for a non-trivial subset of these contended workloads.

Analyzing the impact of system architecture on the scalability of OLTP engines for high-contention workloads

“…• We perform a comparison with other OLTP engine testbeds (DBX1000 [47]) to corroborate our results, and a comparison with the state-of-the-art solutions targeted at high contention (MOCC [43], Orthrus [36], VLL [37]) to tease apart the contribution of each design aspect to overall scalability.…”

“…Recent studies have shown that synchronization overheads imposed by concurrency control protocols are responsible for this lack of scalability [36,43,47]. However, these studies consider only a single system architecture-a non-partitioned, shared-everything one where any transaction can be scheduled to run on any core and can access any data or metadata stored in shared memory.…”

“…Partitioned Serial Execution [23] is another architecture that physically partitions the data across cores such that each core owns a portion of data and serially executes transactions one at a time by using whole-partition locking instead of fine-grained record-level concurrency control. Delegation [2,36] architecture treats a multi-core server like a distributed system. Data and metadata are physically partitioned across threads.…”

“…Thread migration is being used to build virtualization-friendly OLTP engines [12]. Orthrus [36] combines message passing with functional partitioning and transaction pre-execution to improve scalability under high-contention workloads. However, it is unclear how these new architectures compare to the state-of-the-art, or what is the impact of each individual technique on scalability.…”

“…As our work focuses on studying the interaction between process models and concurrency control protocols in the main-memory OLTP context, similarly to recent studies [36,45,47], we restrict our focus to the concurrency control module to avoid confounding effects.…”