Staring into the abyss

Yu, Xiangyao; Bezerra, George; Pavlo, Andrew; Devadas, Srinivas

doi:10.14778/2735508.2735511

Cited by 174 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notes Centralized (deterministic) H-Store [13] two-orders of magnitude YCSB Multi-partition workload Distributed (deterministic) Calvin [18] 22× YCSB Low-contention workload (Uniform access) Centralized (non-deterministic) Cicada [16], TicToc [25], Foudus [15], Ermia [14], Silo [20], 2PL-NoWait [24] 3× TPC-C High-contention workload (1 warehouse) Table 2. Experimental results using TPC-C and YCSB for the centralized implementation of queue-oriented paradigm [17], and a distributed deterministic database.…”

Section: Macrobenchmarkmentioning

confidence: 99%

A queue-oriented transaction processing paradigm

Qadah

2019

Proceedings of the 20th International Middleware Conference Doctoral Symposium

View full text Add to dashboard Cite

Transaction processing has been an active area of research for several decades. A fundamental characteristic of classical transaction processing protocols is non-determinism, which causes them to suffer from performance issues on modern computing environments such as main-memory databases using many-core, and multi-socket CPUs and distributed environments. Recent proposals of deterministic transaction processing techniques have shown great potential in addressing these performance issues. In this position paper, I argue for a queue-oriented transaction processing paradigm that leads to better design and implementation of deterministic transaction processing protocols. I support my approach with extensive experimental evaluations and demonstrate significant performance gains.

show abstract

Section: Macrobenchmarkmentioning

confidence: 99%

A queue-oriented transaction processing paradigm

Qadah

2019

Proceedings of the 20th International Middleware Conference Doctoral Symposium

View full text Add to dashboard Cite

show abstract

“…

…”

mentioning

confidence: 99%

“…Our results show that, with appropriate settings, intelligent scheduling can increase throughput by 54% and reduce abort rate by 80% on a 20-core machine, relative to random scheduling. In summary, the paper provides preliminary evidence that intelligent scheduling significantly improves DBMS performance.Transaction aborts are one of the main sources of performance loss in main memory OLTP systems [35].Current architectures for OLTP DBMS use random scheduling to assign transactions to threads. Random scheduling achieves uniform load across CPU cores and keeps all cores occupied.…”

mentioning

confidence: 99%

Scheduling OLTP transactions via learned abort prediction

Sheng

Tomasic

Zhang

et al. 2019

Proceedings of the Second International Workshop on Exploiting Artificial Intelligence Techniques for Data Management

Self Cite

View full text Add to dashboard Cite

Current main memory database system architectures are still challenged by high contention workloads and this challenge will continue to grow as the number of cores in processors continues to increase [35]. These systems schedule transactions randomly across cores to maximize concurrency and to produce a uniform load across cores. Scheduling never considers potential conflicts. Performance could be improved if scheduling balanced between concurrency to maximize throughput and scheduling transactions linearly to avoid conflicts. In this paper, we present the design of several intelligent transaction scheduling algorithms that consider both potential transaction conflicts and concurrency.To incorporate reasoning about transaction conflicts, we develop a supervised machine learning model that estimates the probability of conflict. This model is incorporated into several scheduling algorithms. In addition, we integrate an unsupervised machine learning algorithm into an intelligent scheduling algorithm. We then empirically measure the performance impact of different scheduling algorithms on OLTP and social networking workloads. Our results show that, with appropriate settings, intelligent scheduling can increase throughput by 54% and reduce abort rate by 80% on a 20-core machine, relative to random scheduling. In summary, the paper provides preliminary evidence that intelligent scheduling significantly improves DBMS performance.Transaction aborts are one of the main sources of performance loss in main memory OLTP systems [35].Current architectures for OLTP DBMS use random scheduling to assign transactions to threads. Random scheduling achieves uniform load across CPU cores and keeps all cores occupied. For workloads with a high abort rate, a large portion of work done by CPU is wasted. In contrast, the abort rate drops to zero if all transactions are scheduled sequentially into one thread. No work is wasted through aborts, but the throughput drops to the performance of a single hardware thread. Research has shown that statistical scheduling of transactions using a history can achieve low abort rate and high throughput [37] for partitionable workloads. We propose a more systematic machine learning approach to schedule transactions that achieves low abort rate and high throughput for both partitionable and non-partitionable workloads. 2 Y. Sheng, et al.The fundamental intuitions of this paper are that (i) the probability that a transactions will abort will high probability can be modeled through machine learning, and (ii) given that an abort is predicted, scheduling can avoid the conflict without loss of response time or throughput. Several research works [1,20,25,27] perform exact analyses of aborts of transaction statements that are complex and not easily generalizable over different workloads or DBMS architectures. Our approach uses machine learning algorithms to model the probability of aborts. Our approach is to (i) build a machine learning model that helps to group transactions that are likely to abort with each ot...

show abstract

“…[5][6][7][8] Most of them are based on the notion of late materialization. Yu et al 10 presented evidences that many-core chip machines require a completely redesigned DBS, which should be modern-hardware aware. Thus, they yield smaller join partial results, decreasing the amount of write operations on secondary memory.…”

mentioning

confidence: 99%

“…However, that premise only holds for a small number of CPU cores in a node. Yu et al 10 presented evidences that many-core chip machines require a completely redesigned DBS, which should be modern-hardware aware. In fact, there are some join operators already which take profit of running on modern-hardware machines.…”

mentioning

confidence: 99%

DaC‐Join: A join operator for improving database performance on modern hardware

Alencar

Brayner

Monteiro

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

The computer industry is moving toward the construction in large scale of chips with hundreds of cores to increase on-chip parallelism. In parallel, computers with solid state memory, the so-called solid state drive (SSD for short), providing petabytes of storage area, is emerging.Nonetheless, database systems (DBSs) were designed based upon two premises. The first is the usage of magnetic disks (HDDs). The second one is that distributed DBSs could scale beyond what a single-node DBMS can support. However, the latter premise only holds for a small number of CPU cores per node and for a limited number of nodes. Thus, to fully exploit benefits of parallelism and high IOPS rates supported by many-core machines with SSDs, database systems should be aware of upcoming CPU architecture and storage technology. In this paper, we present a new join algorithm, denoted DaC-Join, whose key goal is to exploit on-chip parallelism and high IOPS rates delivered by modern hardware. DaC-Join is able to run on multiple cores in parallel, increasing this way intra-operator parallelism. Furthermore, the proposed operator effectively decreases the activity of writing back join temporary results on SSDs, which minimizes the problem of write operation latency in SSDs. In order to validate the novel approach, simulations with a prototype have been conducted over TPC-H benchmark database, scale factor 10. The achieved results evidence the suitability of DaC-Join. KEYWORDS database query processing, parallel join operator, solid state memory 1 INTRODUCTION From a computer hardware perspective, we are witnessing nowadays the existence of two movements toward perhaps a singularity point in computer science. First, the computer industry is moving toward the construction in large scale of chips with hundreds of cores to increase on-chip parallelism. In the near future, we may have several-chip machines, each of which with hundreds of cores. In parallel to the development of several-core chip, a new type of non-volatile memory is emerging, ie, the so-called solid state memory or solid state drive (SSD). Examples of solid state memories are Flash Memory, Phase Change Memory (PCM), Memristors, and Non-Volatile RAM (NV-RAM), among others. SSDs present distinct characteristics and capabilities from hard disks drives (HDDs). The most evident one is the nonexistence of mechanical parts in SSDs. The IOPS rates supported by SSDs may be over 10 2 times greater than 15K RPM HDDs. Write operations on SSDs are much more expensive w.r.t. execution time and energy consumption than read operations phenomenon called read/write asymmetry. A read operation may be up to three times faster and consume up to eight times less energy than write operation. 1 The number of physical write operations on SSD may be far larger than the logical operations, since SSDs internally run two processes (wear leveling and garbage collection), which may induce a rise in the amount of physical write operations. 2 This phenomenon is called write amplification. Another key characteristi...

show abstract

Staring into the abyss

Cited by 174 publications

References 35 publications

A queue-oriented transaction processing paradigm

A queue-oriented transaction processing paradigm

Scheduling OLTP transactions via learned abort prediction

DaC‐Join: A join operator for improving database performance on modern hardware

Contact Info

Product

Resources

About