Adaptive Energy-Aware Scheduling of Dynamic Event Analytics Across Edge and Cloud Resources

Abstract

The growing deployment of sensors as part of Internet of Things (IoT) is generating thousands of event streams. Complex Event Processing (CEP) queries offer a useful paradigm for rapid decision-making over such data sources. While often centralized in the Cloud, the deployment of capable edge devices on the field motivates the need for cooperative event analytics that span Edge and Cloud computing. Here, we identify a novel problem of query placement on edge and Cloud resources for dynamically arriving and departing analytic dataflows. We define this as an optimization problem to minimize the total makespan for all event analytics, while meeting energy and compute constraints of the resources. We propose 4 adaptive heuristics and 3 rebalancing strategies for such dynamic dataflows, and validate them using detailed simulations for 100 − 1000 edge devices and VMs. The results show that our heuristics offer O(seconds) planning time, give a valid and high quality solution in all cases, and reduce the number of query migrations. Furthermore, rebalance strategies when applied in these heuristics have significantly reduced the makespan by around 20 − 25%.

Characterizing application scheduling on edge, fog, and cloud computing resources

“…The energy use may be due to both compute and communication. Edge or fog resources on the field may also run on renewable energy like solar, which have recharge cycles as well that factor into the energy constraint when scheduling …”

“…These have also been extended to include cost/time budgets/goals using functions like GAIN and LOSS . There are greedy algorithms for scheduling BoTs on cloud resources and DAG scheduling on edge devices . They use heuristics such as predicting the proximity between mobile edge devices for deadline planning, prioritizing tasks with the most resource constraints, and incrementally colocating tasks on the same resource to avoid network latency.…”

“…Bittencourt et al scheduled DAGs that are submitted to a cloudlet, and the scheduler can execute it on the “local” or a remote fog, and/or cloud resources. Likewise, Ghosh and Simmhan scheduled transactional DAGs on edge and cloud resources but pinned the source tasks to the edge and the sink tasks to the cloud.…”

“…These can efficiently handle the dynamic task‐based transactional workloads if the user is able to select the right threshold value . Similarly, sensor and event‐based applications have a dynamic workload pattern that schedulers adapt to on edge and fog resources . However, some autoscaling mechanisms use future workload prediction by monitoring current resource utilization to perform proactive autoscaling…”

“…In addition to sensing and generating observation streams, these devices that number in the tens of thousands have spare compute, storage, and memory capacities. These can be leveraged to execute IoT applications at the edge of the network . Further, there has been heightened interest in fog resources, which are between the edge and cloud in the network hierarchy, with compute, storage, and memory capacities that fall between these layers as well .…”