Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach

Abstract

The capability of efficiently processing the data streams emitted by nowadays ubiquitous sensing devices enables the development of new intelligent services. Data Stream Processing (DSP) applications allow for processing huge volumes of data in near real-time. To keep up with the high volume and velocity of data, these applications can elastically scale their execution on multiple computing resources to process the incoming data flow in parallel. Being that data sources and consumers are usually located at the network edges, nowadays the presence of geo-distributed computing resources represents an attractive environment for DSP. However, controlling the applications and the processing infrastructure in such wide-area environments represents a significant challenge. In this paper, we present a hierarchical solution for the autonomous control of elastic DSP applications and infrastructures. It consists of a two-layered hierarchical solution, where centralized components coordinate subordinated distributed managers, which, in turn, locally control the elastic adaptation of the application components and deployment regions. Exploiting this framework, we design several self-adaptation policies, including reinforcement learning based solutions. We show the benefits of the presented self-adaptation policies with respect to static provisioning solutions, and discuss the strengths of reinforcement learning based approaches, which learn from experience how to optimize the application performance and resource allocation.

Designing an efficient clustering strategy for combined Fog-to-Cloud scenarios

“…Moreover, no backup technique is studied. In the work in [30], it is highlighted the relevance of hierarchical and distributed architectures in novel and future IoT-based scenarios, since they improve scalability without compromising stability [30]. Although they consider a hierarchical and decentralized architecture, differently from our proposal their proposal is focused on solving the specific problem of Data Stream Processing, which is different from the FCC problem tackled in this paper, thus resulting in different problem definition, constraints and formulation.…”

“…Indeed, these works could take advantage of our proposal, since they could benefit from deploying their proposed algorithms while delegating control and management issues to the agents to implement the solutions obtained. Moreover, proposals facing architectural aspects, focus on service-specific architectures (e.g., the works in [30] and [31]) and/or consider that roles of the devices in the different layers are known a priori or are assigned according to a less complex criteria than the one considered in our proposal; e.g., coverage range. Aligned with the trends described in the reviewed literature, our work proposes utilizing an optimization technique, i.e., Mixed Integer Linear Programming, to minimize a multi-objective function considering relevant parameters such as latency, energy and mobility; as well as a machine-learning method; i.e., by means of a k-means based algorithm.…”

“…The solution is discarded if no backup leader is found for each cluster. Finally, the cost associated to the subnet is computed using equation 2, line 28, and the lowest cost solution found in all iterations of the k-means procedure (line 14) is considered as the solution for that subnet (lines [29][30][31].…”

Multi-Objective Reinforcement Learning for Reconfiguring Data Stream Analytics on Edge Computing

“…Liu et al [13] propose a single-objective (end-to-end latency) model to reconfigure DSP applications based on collected statistics and Deep Neural Networks. Russo et al [21] offer a multi-objective optimisation (monetary cost and QoS violations) Q-Learning model for reconfiguration. Panerati et al [17] propose an autonomic manager using reinforcement learning algorithms and Q-learning applied to sensors and actuators to provide self-optimisation considering a multi-objective approach.…”

Special Issue on Algorithms for the Resource Management of Large Scale Infrastructures

“…The paper "Multi-level elasticity for wide-area Data Streaming Systems: A reinforcement learning approach" by Russo Gabriele et al [1] addresses the issue of data stream processing (DSP) in the context of a multi cloud/fog. The reference architecture is hierarchically distributed according to the guidelines of the considered Multi-Level Elastic and Distributed DSP Framework (E2DF).…”