Light-Weight Process Groups in the ISIS System

Abstract

A description of ISIS can be found in [7]. In this paper we describe a fundamental element of a new system called HORUS 1 being built at Cornell. HORUS has evolved from ISIS after much experience with building practical fault-tolerant distributed systems.This work was motivated by a trend in the use of ISIS process groups that has become apparent over the last eight years. The process group paradigm is popular with ISIS applications programmers; almost every major application written using ISIS makes extensive use of process groups. In their original design, process groups were intended as a coarse grain transport mechanism for communicating with multiple processes. Process groups were used to represent a replicated service. However, the paradigm has proven popular for more fine grain uses. Over the last few years applications written using ISIS have used process groups to represent objects rather than services. This trend has impacted the original design in several ways and has lead us to focus our attention on providing light-weight process groups.The architecture of HORUS was influenced by microkernel design concepts, in which several light-weight mechanisms are provided in user space. The most obvious of these is the light-weight process or thread abstraction [8,15]. Another well-known, older abstraction is memory allocation. These abstractions not only allow easier resource management by sharing most of a core environment, but also provide a portable interface across different environments.The basic idea behind the light-weight process group (LWG) abstraction is that many LWGs are mapped to a single core group (or set of core groups) as implemented by the kernel of HORUS. Thus, these LWGs share the same security environment (much like threads share the same address space), and the same failure model, while their messages are multiplexed over a single core group transport. The benefit of this approach is that membership changes to the core group automatically affect large numbers of LWGs, amortizing the cost of maintaining membership information over what the application considers a large number of independent groups. The ISIS system lacks such a facility, forcing many application programmers to develop equivalent mechanisms.We have built a prototype of LWGs on top of ISIS V3.0.6 and the initial results show significant improvements in performance. In particular, the LWG subsystem allows LWGs to share the same failure detection protocol execution thereby resulting in faster reaction to member failures and reduced network load. Execution times for typical group operations are also improved: the initial prototype has a speed-up factor of nine for the group create operation (the resulting speed is about SIn Egyptian mythology, HORUS is the son of ISIS. TRENDS IN THE USE OF PROCESS GROUPS3 30 ms), and even higher speed-ups for group joins and leaves.To motivate the problem, we present several examples of how fine grain process groups help solve problems present in distributed applications. We then briefly presen...

Scalable Publish-Subscribe in a Managed Framework

“…In [1], [3] we argue 2 that existing approaches to reliable data dissemination fall into two classes that each scale poorly, although in different senses: (a) systems that run separate protocol instances per topic, like Isis [5], and (b) lightweight group approaches [11] such as Spread [8]. The Isis-like systems can't support large numbers of topics due to the linear per-topic overhead component.…”