Homogeneous Redundancy: a Technique to Ensure Integrity of Molecular Simulation Results Using Public Computing

Abstract

Distributed computing using PCs volunteered by the public can provide high computing capacity at low cost. However, computational results from volunteered PCs have a non-negligible error rate, so result validation is needed to ensure overall correctness. A generally applicable technique is "redundant computing", in which each computation is done on several separate computers, and results are accepted only if there is a consensus. Variations in numerical processing between computers (due to a variety of hardware and software factors) can lead to different results for the same task. In some cases, this can be addressed by doing a "fuzzy comparison" of results, so that two results are considered equivalent if they agree within given tolerances. However, this approach is not applicable to applications that are "divergent", that is, for which small numerical differences can produce large differences in the results. In this paper we examine the problem of validating results of divergent applications. We present a novel approach called Homogeneous Redundancy (HR), in which the redundant instances of a computation are dispatched to numerically identical computers, allowing strict equality comparison of the results. HR has been deployed in Predictor@home, a world-wide community effort to predict protein structure from sequence.

VAWS: Constructing Trusted Open Computing System of MapReduce with Verified Participants

“…Replication and voting features [15] for redundant computing are applied such that multiple computing nodes will perform the same job; the result is accepted when it is submitted by more than half of the total nodes. Sampling techniques address the resource cost of replication, involving result-based sampling [16] and test job injection sampling [17].…”

Performance Prediction and Analysis of BOINC Projects: An Empirical Study with EmBOINC

“…For example, there are a number of criteria for job assignment [2], based on host and job diversity (for example size of the job and speed of the host relative to an estimated statistical distribution, disk and memory requirements for the job to be completed, homogeneous redundancy [23] and host error rate). A scoring-based scheduling policy uses a linear combination of these terms to select the best set of jobs that can be assigned to a given host.…”

Balancing Scientist Needs and Volunteer Preferences in Volunteer Computing Using Constraint Optimization

“…Recently, more sophisticated server-side scheduling policies have been implemented in several BOINC projects. Currently, World Community Grid has a number of criteria for job assignment [2], based on host and job diversity (e.g., size of the job and speed of the host relative to an estimated statistical distribution, disk and memory requirements for the job to be completed, homogeneous redundancy [3] and host error rate). A scoring-based scheduling policy uses a linear combination of these terms to select the best set of jobs that can be assigned to a given host.…”