Ignis: scaling distribution-oblivious systems with light-touch distribution

Abstract

Distributed systems offer notable benefits over their centralized counterparts. Reaping these benefits, however, requires burdensome developer effort to identify and rewrite bottlenecked components. Light-touch distribution is a new approach that converts a legacy system into a distributed one using automated transformations. Transformations operate at the boundaries of bottlenecked modules and are parametrizable by light distribution recipes that guide the intended semantics of the resulting distribution. Transformations and recipes operate at runtime, adapting to load by scaling out only saturated components. Our Ignis prototype shows substantial speedups, attractive elasticity characteristics, and memory gains over full replication, achieved by small and backward-compatible code changes. CCS Concepts • Computer systems organization → Distributed architectures; Cloud computing; • Software and its engineering → Extra-functional properties; Software as a service orchestration system.

Active learning for software engineering

“…Further transformations (not detailed) aim to detect module accesses to their surrounding environment-including accesses to global variables, top-level objects, etc.-and capture a module's full observable behavior. One way to achieve this is by shadowing and transforming free variables in the module environment during the module-loading process [79,81].…”

“…To infer large-scale applications, we propose to exploit the modularity of modern software to enable effective (and scalable) inference. This direction leverages the trend of increasingly popular third-party library usage [6,16,52,79,81,89]. Applications with less modular designs will likely present a cost-of-inference challenge and the need for refining the observation and inference models used.…”