Challenges and Research Directions in Big Data-driven Cloud Adaptivity

Abstract

Abstract:Mainstream cloud technologies are challenged by real-time, big data processing requirements or emerging applications. This paper surveys recent research efforts on advancing cloud computing virtual infrastructures and real-time big data technologies in order to provide dynamically scalable and distributed architectures over federated clouds. We examine new methods for developing cloud systems operating in a real-time, big data environment that can sense the context of the application environment and can adapt the infrastructure accordingly. We describe research topics linked to the challenge of adaptivity such as situation awareness, context detection, service-level objectives, and the capability to predict extraordinary situations requiring remedying action. We also describe research directions for realising adaptivity in cloud computing and we present a conceptual framework that represents research directions and shows interrelations.

Extending TOSCA for Edge and Fog Deployment Support

“…External monitoring mechanisms, e.g., Prometheus (https://prometheus.io), can be used to create an updated "type-level" deployment, which will trigger a reconfiguration of the platform. This workflow is in line with the challenges and research directions for cloud adaptations that were described in our previous paper [30].…”

“…This support can be further extended, by proposing specific modeling for scaling directives, e.g., using TOSCA policies [5]. Based on the scaling directives, it is trivial to automatically create a new TOSCA template, which paves the way for a complete cloud adaptation approach, as described in [30].…”

“…While one may argue that it is difficult to enforce a particular number of instances on a volatile edge topology, we include this capability as it is a business requirement that is very common and that provides to the DevOps the capability to precisely define the requirements of the topology. Further, we assume that the information that is input by the DevOps in a type-level TOSCA template is not static but subject to (automatic) updates based on the state of the processing topology, as illustrated in our previous work [30]. Listing 1 contains an example annotation for a Java class representing the percussion detector fragment (it can either be a placeholder or its actual implementation), which is defined in the illustrative scenario.…”

Severity: A QoS-aware approach to cloud application elasticity

“…Familiar concepts found in traditional rulebased systems, such as aggregations, thresholds and cooldown intervals are still the basic building blocks. Therefore, it also reaps the benefits associated with rule-based adaptivity, such as lower updating overhead, relative genericity with respect to the workload managed, lower computational complexity when compared to other approaches [32]. Moreover, it can automatically use information from a multitude of monitoring attributes which can be provided in each elasticity rule and not only from a limited, hard-coded selection between average CPU, response time and number of requests.…”