RemusDB: transparent high availability for database systems

Abstract

In this paper we present a technique for building a high-availability (HA) database management system (DBMS). The proposed technique can be applied to any DBMS with little or no customization, and with reasonable performance overhead. Our approach is based on Remus, a commodity HA solution implemented in the virtualization layer, that uses asynchronous virtual machine (VM) state replication to provide transparent HA and failover capabilities. We show that while Remus and similar systems can protect a DBMS, database workloads incur a performance overhead of up to 32% as compared to an unprotected DBMS. We identify the sources of this overhead and develop optimizations that mitigate the problems. We present an experimental evaluation using two popular database systems and industry standard benchmarks showing that for certain workloads, our optimized approach provides very fast failover (≤ 3 seconds of downtime) with low performance overhead when compared to an unprotected DBMS. Our approach provides a practical means for existing, deployed database systems to be made more reliable with a minimum of risk, cost, and effort. Furthermore, this paper invites new discussion about whether the complexity of HA is best implemented within the DBMS, or as a service by the infrastructure below it.

A Survey on Resiliency Techniques in Cloud Computing Infrastructures and Applications

“…In this regard, similarity compression (SC) [76] reduces the storage space by using a compression algorithm, while fast and capacity-efficient VM checkpoint (FVMCheck) [68] reduces the storage space and synchronization overhead using page-caching technique. More recently, an extension of Remus, RemusDB [69], has been proposed to reduce synchronization and checkpoint overhead (see Table I). …”

Tradeoffs in compressing virtual machine checkpoints

“…Delta compression identifies the parts in a dirty page that are changed when the page is written to, i.e., the page delta, and replicates the delta to the backup instead of the entire page. It has been used in a few HA systems [21,24].…”

“…One reduces the amount of VM state to protect/checkpoint. RemusDB [21], a highly available database system in VM, does not checkpoint clean disk buffers, and "de-protects" certain data structures in the database system that can be regenerated after a failure. This creates smaller checkpoints and thus less replication traffic in the network.…”

“…The authors of [13] briefly discussed compressing checkpoints by gzip and delta compression, although a thorough evaluation was not included. RemusDB [21] and SecondSite [24], designed for database HA and datacenter disaster recovery, respectively, use delta compression in their systems. They both find page delta by XOR, and compress the delta by RLE, like evaluated in this paper.…”

“…It can be done by a general-purpose tool, such as gzip. Alternatively, for each dirty memory page in a checkpoint, the bits that are actually changed (called the page delta) may be identified, and only the delta is replicated [21,24,27,31]. These compression methods are available, but they have not been compared systematically.…”