Tackling Latency via Replication in Distributed Systems

Abstract

Consistently high reliability and low latency are twin requirements common to many forms of distributed processing; for example, server farms and mirrored storage access. To address them, we consider replication of requests with canceling -i.e. initiate multiple concurrent replicas of a request and use the first successful result returned, canceling all outstanding replicas. This scheme has been studied recently, but mostly for systems with a single central queue, while server farms exploit distributed resources for scalability and robustness. We develop an approximate stochastic model to determine the response-time distribution in a system with distributed queues, and compare its performance against its centralized counterpart. Validation against simulation indicates that our model is accurate for not only the mean response time but also its percentiles, which are particularly relevant for deadline-driven applications. Further, we show that in the distributed set-up, replication with canceling has the potential to reduce response times, even at relatively high utilization. We also find that it offers response times close to those of the centralized system, especially at medium-to-high request reliability. These findings support the use of replication with canceling as an effective mechanism for both fault-and delay-tolerance.

Extending Shared-Memory Computations to Multiple Distributed Nodes

“…In pseudo-replication, concurrent instances of an executable are asynchronously launched on multiple nodes in the cluster with each instance executing to create a distinct portion of output. Although the AMNE behavior may initially appear to be similar to task replication used in distributed systems [32], [33], [34], [35], it differs in many aspects as described below: 1) In task replication, all replicated instances of the executable are identical and they work on the same data. The job scheduler in the cluster launches the same executable multiple times.…”

“…3) The primary objective of task replication is to address latency in nodes, to improve quality of service or to improve application fault tolerance [34]. Parallelization of tasks is not an objective.…”

SHAM: Scalable Homogeneous Addressing Mechanism for structured P2P networks

“…Replication techniques play a major role in reducing latency, improving load balancing, and enhancing availability [21][22][23][24]. Some known techniques are synchronous, asynchronous, dynamic, full, and neighborhood replication [25][26][27].…”

Modeling of Request Cloning in Cloud Server Systems using Processor Sharing

“…They modeled servers using M/M/1 queues, i.e., queues where the arrivals follow a Poisson process and job service times have an exponential distribution. Other notable contributions concerning cloning with exponential distributions include [3,10,20]. Qiu et al [20] compares the use of multiple queues (in a distributed servers setting) to a central queue.…”

“…Other notable contributions concerning cloning with exponential distributions include [3,10,20]. Qiu et al [20] compares the use of multiple queues (in a distributed servers setting) to a central queue. Gardner et al [10] derived results on the largest marginal improvement that can be obtained using the Redundancy-d cloning policy, that clones each request to exactly d servers.…”