Ephemeral Partially Replicated Databases

Abstract

In today's analytics-driven world, fully replicated isolated databases provide much-needed database availability and compute scalability but at the cost of storage scalability, an issue that is addressed by partially replicated isolated databases. However, a partially replicated database that is optimal at the time of design is soon made inefficient by changing business needs, products & services it offers, datasets and query workloads. To this address this issue, we introduce the notion of migration cost as an additional factor that influences the design of a partially replicated databases. In this paper, we formalize the notion of migration cost and present a new cost-based objective function to partition and allocate data elements across available databases. Further, we discuss its implementation in the context of Uber and demonstrate its effectiveness based on a 10-week simulation study.