Pleiades: Distributed Structural Invariants at Scale

Abstract

Modern large scale distributed systems increasingly espouse sophisticated distributed architectures characterized by complex distributed structural invariants. Unfortunately, maintaining these structural invariants at scale is time consuming and error prone, as developers must take into account asynchronous failures, loosely coordinated subsystems and network delays. To address this problem, we propose PLEIADES, a new framework to construct and enforce large-scale distributed structural invariants under aggressive conditions. PLEIADES combines the resilience of self-organizing overlays, with the expressiveness of an assembly-based design strategy. The result is a highly survivable framework that is able to dynamically maintain arbitrary complex distributed structures under aggressive crash failures. Our evaluation shows in particular that PLEIADES is able to restore the overall structure of a 25,600 node system in less than 11 asynchronous rounds after half of the nodes have crashed.

Computation Against a Neighbour: Addressing Large-Scale Distribution and Adaptivity with Functional Programming and Scala

“…The Origami Shape Language defined in [Nag08] is used to achieve similar goals of the Growing Point Language though focussing on programming a "computational surface", intended as a set of small devices working independently of their density in the surface: this language defines geometrical constructs to create basic regions and compose them, which could be turned into an API of NC/ScaFi blocks to be functionally composed to achieve similar complex geometrical structures. A more recent work is PLEIADES [BBLT18], which aim at expressing self-stabilising overlay structures in large-scale distributed systems; however, these goals are not achieved through programs as in NC/ScaFi but rather through shape formation protocols and equation-based specifications. In general, due to the universal character of field computations [ABDV18], one could consider NC/ScaFi as a viable implementation framework for a number of approaches to organise the shape of computational entities in a physical environment, with the additional byproduct of leveraging the theory of field computations to assess formal validity of certain properties, such as density independence as developed in [BVPD17] or self-stabilisation in [VAB + 18].…”

Macroprogramming: Concepts, State of the Art, and Opportunities of Macroscopic Behaviour Modelling

“…The creation of macro-structures is sometimes a goal of macroprogramming (cf. topology programming in Pleiades (Bouget et al, 2018)). • The macro-behavioural view considers behaviours emerging from multiple components of a system.…”

Engineering collective intelligence at the edge with aggregate processes

“…Examples include Growing Point Language (GPL) [63], which uses a botanical metaphor with growth processes replicating tropisms in plants, and Origami Shape Language (OSL) [64], which provides geometrical constructs to create and compose regions of a "computational surface". A more recent framework is Pleiades [65], a topology programming framework which exploits self-organising overlays and assembly-based modularity to construct and enforce self-stabilising structural invariants. These features can be useful, e.g., for morphogenesis and self-assembling robots.…”