Self-Adapting Reliability in Distributed Software Systems

Abstract

Developing modern distributed software systems is difficult in part because they have little control over the environments in which they execute. For example, hardware and software resources on which these systems rely may fail or become compromised and malicious. Redundancy can help manage such failures and compromises, but when faced with dynamic, unpredictable resources and attackers, the system reliability can still fluctuate greatly. Empowering the system with self-adaptive and self-managing reliability facilities can significantly improve the quality of the software system and reduce reliance on the developer predicting all possible failure conditions. We present iterative redundancy, a novel approach to improving software system reliability by automatically injecting redundancy into the system's deployment. Iterative redundancy self-adapts in three ways: (1) by automatically detecting when the resource reliability drops, (2) by identifying unlucky parts of the computation that happen to deploy on disproportionately many compromised resources, and (3) by not relying on a priori estimates of resource reliability. Further, iterative redundancy is theoretically optimal in its resource use: Given a set of resources, iterative redundancy guarantees to use those resources to produce the most reliable version of that software system possible; likewise, given a desired increase in the system's reliability, iterative redundancy guarantees achieving that reliability using the least resources possible. Iterative redundancy handles even the Byzantine threat model, in which compromised resources collude to attack the system.We evaluate iterative redundancy in three ways. First, we formally prove its self-adaptation, efficiency, and optimality properties. Second, we simulate it at scale using discrete event simulation. Finally, we modify the existing, open-source, volunteer-computing BOINC software system and deploy it on the globally-distributed PlanetLab testbed network to empirically evaluate that iterative redundancy is self-adaptive and more efficient than existing techniques.

eQual: informing early design decisions

“…It is likely that other tools that provide other types of information useful for making design decisions can be complementary to eQual, as Section 5 describes. This paper provides significant evidence that eQual can be a part of the solution to [11,12].…”

“…We use a simplified model of Hadoop from prior work [11,12] as a running example throughout the paper. In this model (shown in Figure 1), a computation is the problem being solved, consisting of many computational tasks.…”

“…In this model (shown in Figure 1), a computation is the problem being solved, consisting of many computational tasks. Tasks can be replicated, e.g., for reliability [12]. A job is an instance of a task (i.e., one replica of a task) and machines execute these jobs.…”

“…We will discuss other, non-numerical variation points below. Figure 2 depicts representative bounds for the five variation points obtained from the previously published analysis [11,12].…”

Challenges in Composing and Decomposing Assurances for Self-Adaptive Systems

“…In modern software systems, self-adaptation can take place both via mechanisms integrated in individual components as well as in groups/collectives of components (e.g., [8][9][10][11][12]), and that have the goal of modifying something in the behaviour of a component or a collective (e.g., [6,7]). …”