Using restricted transactional memory to build a scalable in-memory database

Abstract

The recent availability of Intel Haswell processors marks the transition of hardware transactional memory from research toys to mainstream reality. DBX is an in-memory database that uses Intel's restricted transactional memory (RTM) to achieve high performance and good scalability across multicore machines. The main limitation (and also key to practicality) of RTM is its constrained working set size: an RTM region that reads or writes too much data will always be aborted. The design of DBX addresses this challenge in several ways. First, DBX builds a database transaction layer on top of an underlying shared-memory store. The two layers use separate RTM regions to synchronize shared memory access. Second, DBX uses optimistic concurrency control to separate transaction execution from its commit. Only the commit stage uses RTM for synchronization. As a result, the working set of the RTMs used scales with the meta-data of reads and writes in a database transaction as opposed to the amount of data read/written. Our evaluation using TPC-C workload mix shows that DBX achieves 506,817 transactions per second on a 4-core machine.

In-Memory Big Data Management and Processing: A Survey

“…The two types of transactional memory, i.e., software transactional memory (STM) and hardware transactional memory (HTM), are compared in Table 2. STM causes a significant slowdown during execution and thus has limited practical application [194], while HTM has attracted new attention for its efficient hardwareassisted atomic operations/transactions, since Intel introduced it in its mainstream Haswell microarchitecture CPU [102], [103]. Haswell HTM is implemented based on cache coherency protocol.…”

“…The author [102] exploits HTM based on HLE, by dividing a database transaction into a set of relatively small HTM transactions with timestamp ordering (TSO) concurrency control and minimizing the false abort probability via data/index segmentation. RTM is utilized in [103], which uses a threephase optimistic concurrency control to coordinate a whole database transaction, and protects single data read (to guarantee consistency of sequence numbers) and validate/write phases using RTM transactions.…”

“…Hekaton [104], [107] utilizes optimistic MVCC and lock-free data structures to achieve high concurrency efficiently. Besides, hardware transactional memory (HTM) [102], [103] is being increasingly exploited in concurrency control for OLTP. Query processing.…”

Scaling HTM-Supported Database Transactions to Many Cores

“…Wang et al [42] combine Haswell's HTM with optimistic concurrency control to build a scalable in-memory database systems. Their approach similar to ours, but requires a final commit phase that is executed in a single hardware transaction and which encompasses the meta data of the transactions' read and write set.…”

Applying HTM to an OLTP System

“…A recent study has demonstrated that TSX can improve performance of operations on common tree index structures [10]. Finally, a recent proposal demonstrates that a design tuned for Intel's RTM instructions can offer performance comparable to a stateof-the-art main memory transaction processing system with fine-grained locks while having lower code complexity [21]. Our study is complementary to these results, since we use a complete complex transaction processing system as a starting point and evaluate the applicability of HTM without redesigning any of the components.…”