Hybrid Workflow Provisioning and Scheduling on Edge Cloud Computing Using a Gradient Descent Search Approach

Alsurdeh, Raed; Calheiros, Rodrigo N.; Matawie, Kenan M; Javadi, Bahman

doi:10.1109/ispdc51135.2020.00019

Cited by 8 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Batch processing involves storing the upcoming data before processing while streaming processing is related to performing operations on data streams in a realtime or near-time manner. We refer to this integration as a hybrid workflow [9]. Hybrid workflows can be applied in many application domains like traffic monitoring [10], social sensing [11], and business analytic [12].…”

Section: Introductionmentioning

confidence: 99%

“…The motivation of hybrid model is to provide an efficient management and control over applications which are complex in scale and internal dependencies, involve short-term stream intervals and online batch feeding, and flexible to their parameters tuning. In this paper, we are extending our previous work [9] to propose a hybrid workflow scheduling framework on edge cloud computing that considers the integration requirements for hybrid workflows while optimizing the workflow execution time and monetary cost. The framework involves algorithms for resource estimation, provisioning, and task scheduling.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hybrid Workflow Scheduling on Edge Cloud Computing Systems

Alsurdeh¹,

Calheiros

Matawie

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Internet of Things applications can be represented as workflows in which stream and batch processing are combined to accomplish data analytics objectives in many application domains such as smart home, health care, bioinformatics, astronomy, and education. The main challenge of this combination is the differentiation of service quality constraints between batch and stream computations. Stream processing is highly latency-sensitive while batch processing is more likely resource-intensive. In this work, we propose an end-to-end hybrid workflow scheduling on an edge cloud system as a two-stage framework. In the first stage, we propose a resource estimation algorithm based on a linear optimization approach, gradient descent search (GDS), and in the second stage, we propose a cluster-based provisioning and scheduling technique for hybrid workflows on heterogeneous edge cloud resources. We provide a multi-objective optimization model for execution time and monetary cost under constraints of deadline and throughput. Results demonstrate the framework performance in controlling the execution of hybrid workflows by efficiently tuning several parameters including stream arrival rate, processing throughput, and workflow complexity. In comparison to a meta-heuristics technique using Particle Swarm Optimization (PSO), the proposed scheduler provides significant improvement for large-scale hybrid workflows in terms of execution time and cost with an average of 8% and 35%, respectively.

show abstract