Beyond the mean in fork-join queues: Efficient approximation for response-time tails

Tackling Latency via Replication in Distributed Systems

“…, dr−1), where di=qi+1−qi for 1≤i≤r−1. Notice that this model is closely related to the fork-join model in [18], where the difference in queue-lengths are also used to model the evolution of a set of queues. However, here we consider the replication with canceling mechanism, which displays different dynamics from the fork-join queue, and the replicas are allowed to fail, a feature not considered for the forkjoin queue.…”

“…However, here we consider the replication with canceling mechanism, which displays different dynamics from the fork-join queue, and the replicas are allowed to fail, a feature not considered for the forkjoin queue. Also, [18] relies on the queue-length differences with respect to the shortest queue, while here we focus on the differences between two consecutive queues after ordering. The queue-length difference is unbounded in principle, but, to keep the phase space finite, we introduce an upper bound C<∞, such that the difference is at most C. As a result, the service process D(t) takes values in the set SD = {(d1, .…”

“…Thus, during the not-all-busy period we keep track of the arrival and services phases J(t)=(A(t), D(t)), with the service phase D(t) taking values in the set SD. We can thus follow [18] to find the stationary distribution π(0) that solves…”

“…Further, the phase in which the service stage ends determines the stage in which the waiting stage begins. The remaining of the proof follows the same steps as that of [18,Theorem 1], such that the matrixPs,w=Γ −1 (T −S (MAP) ) Λ is a stochastic matrix that determines how the phase at the end of the service phase determines the phase at the beginning of the waiting phase. Once a job starts the waiting phase, it evolves according to Swait until absorption.…”

“…Once a job starts the waiting phase, it evolves according to Swait until absorption. Further details can be found in [18].…”