Distributed backup scheduling: Modeling and optimization

Abstract

Abstract-Recent years have seen rapid growth in data storage, magnifying the importance of ensuring data safety by performing regular backups. However, traffic created by such backups can be a significant burden on the underlying communication network. In the present paper we address the tradeoff between frequent backups (increased safety) and reducing the network peak load. We address the problem of shifting backup traffic from peak hours to off-peak hours within the constraints imposed by user connectivity. Backups are scheduled using a distributed protocol characterized by a set of probabilities that indicate the likelihood of a user initiating a backup during a given hour. Given these probabilities, we study the network capacity by investigating the rate at which users can generate data while retaining stable backlog processes. We then derive explicit expressions for the stationary behavior of the backup process, and discuss how to choose the backup probabilities that strike the right balance between a low peak load and data safety. Via simulation experiments we show that this approach is highly successful in reducing costs.

A queueing-theoretic analysis of the threshold-based exhaustive data-backup scheduling policy

“…This paper is one of the first to address the data backup challenge from a queueing-theoretic point of view. To the best of our knowledge, only van de Ven et al also consider this point of view [4]. Whereas we study a threshold-based policy, van de Ven et al examine a probabilistic backup policy, that is at each time slot the user decides with some probability whether to initiate a backup, regardless of the backlog size.…”

“…There are several directions for future research. The assumption of iid sequence {A k } may be relaxed, because in practice it is likely that the arrival process exhibits some cyclic pattern, as described by van de Ven et al [4]. Our approach will then have to be slightly adapted by using partial or vector generating functions instead of probability generating functions, in order to keep track of the position within the cycle.…”

“…We are only aware of the paper of van de Ven et al [4], where a discrete-time queueing model for a distributed backup scheduling protocol for corporate networks with homogeneous users (i.e., same data production and connectivity patterns) is studied. Particular attention was paid to the tradeoff between frequent backups and reducing the network peak load, taking into account connectivity patterns of users.…”

Self-service data protection for stateful containers

“…RPO compliance has also been studied in non-containerized environments. For instance, leveraging historic information [11], or accounting for existing traffic patterns [16] or criticality of data being backed up [18] help in complying with RPO. However, the backup environment may still experience unexpected failures or resource overload, which is more typical in containerized environments due to high job churn.…”

“…rate of change, IO burst patterns, and in a cluster undergoing various bandwidth patterns over a long period of simulated time. Use of simulators [10,16] is common to evaluate schedulers at a large scale. Evaluation with 2500 volumes with varying RPOs shows that our system can quickly react to resource fluctuations and reduce RPO violations and required snapshots.…”