Self-Organizing Scheduling on the Organic Grid

Abstract

Abstract-We propose a biologically inspired and fullydecentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. Our approach achieves the following design objectives: near-zero knowledge of network topology, zero knowledge of system status, autonomous scheduling, distributed computation, lack of specialized nodes. Every node is equally responsible for scheduling and computation, both of which are performed with practically no information about the system.We believe that this model is ideally suited for large-scale unstructured grids such as desktop grids. This model avoids the extensive system knowledge requirements of traditional Grid scheduling approaches. Contrary to the popular master/worker organization of current desktop grids, our approach does not rely on specialized super-servers or on application-specific clients. By encapsulating computation and scheduling behavior into mobile agents, we decouple both application code and scheduling functionality from the underlying infrastructure. The resulting system is one where every node can start a large grid job, and where the computation naturally organizes itself around available resources.Through the careful design of agent behavior, the resulting global organization of the computation can be customized for different classes of applications. In a previous paper, we described a proof-of-concept prototype for an independent task application. In this paper, we generalize the scheduling framework and demonstrate that our approach is applicable to a computation with a highly synchronous communication pattern, namely Cannon's matrix multiplication.

Revisiting Matrix Product on Master-Worker Platforms

“…Self-optimization methodologies are described by Cuenca et al [21] and by Chen et al [16]. Along the same line, Chakravarti et al [15] describe an implementation of Cannon's algorithm using self-organizing agents on a peer-to-peer network.…”

Multi-level reconfigurable self-organization in overlay services

“…Their properties, e.g., path uniqueness, and the hierarchy they reflect enable operations such as decentralized search, decision-making, aggregation or information dissemination. These operations are fundamental in various application domains such as distributed databases [González-Beltrán et al, 2008, Zhuge andFeng, 2008], application-level multimedia multicasting [Tan et al, 2005a] and grid computing [Chakravarti et al, 2006].…”

“…Principal operations such as information dissemination, decision-making, aggregation, lookup and search have been studied as overlay-based solutions [Zhuge and Feng, 2008, Chakravarti et al, 2006, Gkantsidis et al, 2006, Jelasity et al, 2007, Van Renesse et al, 2003. The aforementioned emerging application domains become more information-driven, cyber-physical, ambient and heterogeneous and therefore have similar operational requirements that can be met by overlay networks.…”

Redundant movements in autonomous mobility: Experimental and theoretical analysis

“…it knows key information like remaining execution time and program size. Another difference is that the approach does not split a program into subtasks as in [10,25,26]. An AMP is the whole program.…”