An $O(N)$ Sorting Algorithm: Machine Learning Sort

Zhao, Haitao; Luo, Yuehan

doi:10.48550/arxiv.1805.04272

Cited by 2 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the area of databases, examples include cardinality estimation [25,29,45,62], join order planning [27,37,44] and configuration tuning [58]. Besides databases, works have been done to improve buffer management systems [4], sorting algorithms [73], memory page prefetching [19,71] and memory control [21], task scheduling [26], CPU scheduling [51], locking priority [12] and cache replacement [52]. Although these works try to leverage machine learning to make systems self-aware, but none of them targets on the concurrency control.…”

Section: Related Workmentioning

confidence: 99%

Polyjuice: High-Performance Transactions via Learned Concurrency Control

Wang,

Ding,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Concurrency control algorithms are key determinants of the performance of in-memory databases. Existing algorithms are designed to work well for certain workloads. For example, optimistic concurrency control (OCC) is better than twophase-locking (2PL) under low contention, while the converse is true under high contention.To adapt to different workloads, prior works mix or switch between a few known algorithms using manual insights or simple heuristics. We propose a learning-based framework that instead explicitly optimizes concurrency control via offline training to maximize performance. Instead of choosing among a small number of known algorithms, our approach searches in a "policy space" of fine-grained actions, resulting in novel algorithms that can outperform existing algorithms by specializing to a given workload.We build Polyjuice based on our learning framework and evaluate it against several existing algorithms. Under different configurations of TPC-C and TPC-E, Polyjuice can achieve throughput numbers higher than the best of existing algorithms by 15% to 56%.

show abstract

Section: Related Workmentioning

confidence: 99%

Polyjuice: High-Performance Transactions via Learned Concurrency Control

Wang,

Ding,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In the area of database, examples include cardinality estimation [30,25,53,46], join order planning [28,37,43] and configuration tuning [52]. Besides database, works have been done to improve buffer management systems [10], sorting algorithms [58], memory page prefetching [19,56] and memory controller [23] and scheduling [26]. Many of these scenarios face similar challenges of dealing with shifting data distribution, which could be other applications of our model caching mechanism.…”

Section: Related Workmentioning

confidence: 99%

Learned Indexes for Dynamic Workloads

Tang¹,

Zhou²,

Wang³

et al. 2019

Preprint

View full text Add to dashboard Cite

The recent proposal of learned index structures opens up a new perspective on how traditional range indexes can be optimized. However, the current learned indexes assume the data distribution is relatively static and the access pattern is uniform, while real-world scenarios consist of skew query distribution and evolving data. In this paper, we demonstrate that the missing consideration of access patterns and dynamic data distribution notably hinders the applicability of learned indexes. To this end, we propose solutions for learned indexes for dynamic workloads (called Doraemon). To improve the latency for skew queries, Doraemon augments the training data with access frequencies. To address the slow model re-training when data distribution shifts, Doraemon caches the previously-trained models and incrementally fine-tunes them for similar access patterns and data distribution. Our preliminary result shows that, Doraemon improves the query latency by 45.1% and reduces the model re-training time to 1/20.

show abstract

An $O(N)$ Sorting Algorithm: Machine Learning Sort

Cited by 2 publications

References 13 publications

Polyjuice: High-Performance Transactions via Learned Concurrency Control

Polyjuice: High-Performance Transactions via Learned Concurrency Control

Learned Indexes for Dynamic Workloads

Contact Info

Product

Resources

About