An in-memory object caching framework with adaptive load balancing

Abstract

The extreme latency and throughput requirements of modern web applications are driving the use of distributed inmemory object caches such as Memcached. While extant caching systems scale-out seamlessly, their use in the cloud -with its unique cost and multi-tenancy dynamicspresents unique opportunities and design challenges.In this paper, we propose MBal, a high-performance inmemory object caching framework with adaptive Multiphase load Balancing, which supports not only horizontal (scale-out) but vertical (scale-up) scalability as well. MBal is able to make efficient use of available resources in the cloud through its fine-grained, partitioned, lockless design. This design also lends itself naturally to provide adaptive load balancing both within a server and across the cache cluster through an event-driven, multi-phased load balancer. While individual load balancing approaches are being leveraged in in-memory caches, MBal goes beyond the extant systems and offers a holistic solution wherein the load balancing model tracks hotspots and applies different strategies based on imbalance severity -key replication, server-local or cross-server coordinated data migration. Performance evaluation on an 8-core commodity server shows that compared to a state-of-the-art approach, MBal scales with number of cores and executes 2.3× and 12× more queries/second for GET and SET operations, respectively.

Erasure coding for small objects in in-memory KV storage

“…We point out that MemEC has comparable performance to existing in-memory storage systems proposed in the literature (e.g., [19,30,37]). For example, in a commodity cluster over a Gigabit network, their access latencies are reportedly in the range of few milliseconds.…”

Elastic Provisioning of Cloud Caches

“…Figure 9 (right) shows that sometimes servers need to respond to 30% more traffic than the average. Rebalancing the server load can be done with known techniques that keeps track of the highly loaded slot and balance them among the servers, such as the ones presented in [23] and [17].…”

KeyValueServe†: Design and performance analysis of a multi‐tenant data grid as a cloud service

“…Existing services such as S3 and DynamoDb 28,30 GD-Wheel 24 and CAMP 23 propose new key replacement strategies in caches to have high cache hits based on the recency or frequency and re-computation costs, completely orthogonal to our objectives. MBal, 20 CloudScale, 50 and SCADS 51 perform cluster load balancing mitigating hot-spots through key replication or data migration differing from our work. Argon, 52 Walraven et al, 53 EyeQ, 54 SQLVM, 55 Pulsar, 21 Das et al, 56 IOFlow, 57 Zeng and Plale, 58 Existing database and key-value store services perform sharing at a much coarser granularity, leaving scopes of resources under-utilization and resulting in lower degrees of concurrent accesses.…”