Reproducible testing of distributed software with middleware virtualization and simulation

Abstract

P2P Grids are Grids organized into P2P networks where participant exchange computing time so as to complete computational tasks. Evaluating the performance of scheduling algorithms enables one to deploy those that are efficient. Performance is often evaluated experimentally or through simulation because these algorithms (typically heuristics) are too complex to model analytically. Testing the implementation of P2P Grid middleware before it is deployed is also important: Reproducing configurations or conditions that lead to unexpected outcomes is thus valuable.A P2P Grid environment exhibits multiple sources of failure and is typically dynamic and uncontrollable. Reproducing even basic behavior of Grid nodes in a controllable and repeatable manner is thus exceedingly difficult. Such lack of control over the environment is a major challenge in the software engineering of P2P Grid middleware [7]. Simulators have been proposed to evaluate the performance of scheduling algorithms, but are often limited in scope, reusability and accuracy, i.e. they rely on simplified models.We introduce a software engineering pattern -that we call code once, deploy twice -to both reduce the distance between simulated and implemented algorithms and reproduce, at will, Grid configurations and environments: A simulator implementation of a Grid architecture is built by virtualizing its middleware implementation. An immediate benefit is that most of the code can be reused between both implementations; only communications between Grid nodes, multithreading within Grid nodes and actual task execution are coded differently. As a derived benefit, most of the code of the middleware can be tested within the controlled environment of the simulator, before it is deployed as-is. Another benefit is high simulation accuracy. We describe the implementation of a P2P Grid following the code once, deploy twice pattern, that we believe is also relevant to other Grid types (certainly Volunteer Grids [5,4] and Desktop Grids [22], and possibly Globus-based Grids [3]).