Placement is not Enough: Embedding with Proactive Stream Mapping on the Heterogenous Edge

Abstract

Edge computing is naturally suited to the applications generated by Internet of Things (IoT) nodes. The IoT applications generally take the form of directed acyclic graphs (DAGs), where vertices represent interdependent functions and edges represent data streams. The status quo of minimizing the makespan of the DAG motivates the study on optimal function placement. However, current approaches lose sight of proactively mapping the data streams to the physical links between the heterogenous edge servers, which could affect the makespan of DAGs significantly. To solve this problem, we study both function placement and stream mapping with data splitting simultaneously, and propose the algorithm DPE (Dynamic Programmingbased Embedding). DPE is theoretically verified to achieve the global optimality of the embedding problem. The complexity analysis is also provided. Extensive experiments on Alibaba cluster trace dataset show that DPE significantly outperforms two state-of-the-art joint function placement and task scheduling algorithms in makespan by 43.19% and 40.71%, respectively.

Workflow Makespan Minimization for Partially Connected Edge Network: A Deep Reinforcement Learning-Based Approach

“…Specifically, within a given workflow, the task with no predecessor is considered the entry task 𝑣 𝑒𝑛𝑡𝑟𝑦 , while the task with no successor is called the exit task 𝑣 𝑒𝑥𝑖𝑡 . For workflows with multiple-entry or multiple-exit tasks, a pseudo entry task 𝑣 𝑝−𝑒𝑛𝑡 and a pseudo exit task 𝑣 𝑝−𝑒𝑥𝑖𝑡 are added to the predecessor of 𝑣 𝑒𝑛𝑡𝑟𝑦 and successor of 𝑣 𝑒𝑥𝑖𝑡 respectively [19,23,26]. Since 𝑣 𝑝−𝑒𝑛𝑡 and 𝑣 𝑝−𝑒𝑥𝑖𝑡 are tasks with zero execution and transmission costs, this modification does not affect the scheduling process.…”

“…There are already studies on router level multi-path routing techniques [27][28][29], and the advancement in segment routing further proves its practical feasibility [29]. Inconsistent with the basic assumptions in [23], we consider a scenario where task 𝑣 𝑙 is executed on 𝑛 𝑖 and one of the task's successor tasks 𝑣 𝑚 is scheduled to 𝑛 𝑗 , denoting 𝓅 ∈ 𝒫 𝑖,𝑗 as one of the subpaths, and 𝛼 𝓅 denotes the ratio of the respective segment. The segment of the data stream from task 𝑣 𝑙 to 𝑣 𝑚 in subpath 𝓅 is hereby defined as Eq.…”

“…When the placement of adjacent tasks is determined, the upcoming problem is to find proper routing paths for data stream transmission. First, we use the RPF algorithm proposed in [23] to find all the possible paths from two edge servers. However, these paths are hardly to be disjoint, thus leading to the potential risk of congestion during the multipath routing phase.…”

“…Inconsistent with the network setup in [23], in our simulation, we set 10 edge servers in the fully connected edge network with their respective processing power randomly sampled at [20,40] Gflop/s, while the bandwidth is randomly sampled at [30,80] Mbit/s as the benchmark network. All of the simulations are performed on a desktop with an I7-7700K CPU operating at 4.2GHz and 16GB memory.…”

“…With recent advancements in network management techniques such as network function virtualization (NFV) and softwaredefined networks (SDNs), multipath routing techniques have received growing attention both academically and industrially, not only because these techniques provide backup plans when link outages occur [20] but also because these techniques provide an emerging solution to improve network efficiency [21,22]. In [23], an optimization algorithm was proposed to solve the task scheduling and multipath data stream routing of a workflow by using dynamic programming. However, the study failed to determine the dispatching order of the tasks, and ignoring the congestion of the same task during the multipath transmission further degraded the performance of the algorithm.…”