Decentralized Edge-to-Cloud Load Balancing: Service Placement for the Internet of Things

Abstract

The Internet of Things (IoT) requires a new processing paradigm that inherits the scalability of the cloud while minimizing network latency using resources closer to the network edge. On the one hand, building up such flexibility within the edge-to-cloud continuum consisting of a distributed networked ecosystem of heterogeneous computing resources is challenging. On the other hand, IoT traffic dynamics and the rising demand for low-latency services foster the need for minimizing the response time and a balanced service placement. Load-balancing for fog computing becomes a cornerstone for cost-effective system management and operations. This paper studies two optimization objectives and formulates a decentralized load-balancing problem for IoT service placement: (global) IoT workload balance and (local) quality of service (QoS), in terms of minimizing the cost of deadline violation, service deployment, and unhosted services. The proposed solution, EPOS Fog, introduces a decentralized multi-agent system for collective learning that utilizes edge-to-cloud nodes to jointly balance the input workload across the network and minimize the costs involved in service execution. The agents locally generate possible assignments of requests to resources and then cooperatively select an assignment such that their combination maximizes edge utilization while minimizes service execution cost. Extensive experimental evaluation with realistic Google cluster workloads on various networks demonstrates the superior performance of EPOS Fog in terms of workload balance and QoS, compared to approaches such as First Fit and exclusively Cloud-based. The results confirm that EPOS Fog reduces service execution delay up to 25% and the load-balance of network nodes up to 90%. The findings also demonstrate how distributed computational resources on the edge can be utilized more cost-effectively by harvesting collective intelligence.INDEX TERMS Agent, cloud computing, collective learning, distributed optimization, edge computing, fog computing, internet of things (IoT), load-balancing, service placement.

Fusion of IoT, AI, Edge–Fog–Cloud, and Blockchain: Challenges, Solutions, and a Case Study in Healthcare and Medicine

“…This means honing in on the most appealing data by carefully selecting only those data/information of interest from a noisy environment with limited resources [21], [57] • The role of AI in the IoT Infrastructures: Another layer of complication with IoT is the infrastructures. Indeed, there are many aspects and subcomponents to IoT infrastructures that AI is integrated as analyzing the deluge of constraints (e.g., energy consumption, latency, QoS) and data generated by IoT infrastructures that otherwise was simply impossible for humans to review and extract insights.The following are some of the most common benefits of incorporating AI into the IoT network/infrastructures layer: security (e.g., tracking down a wide variety of cyber threats from malware to phishing attacks, breach risk prediction, and fogassisted endpoint protection, among others), optimization of connectivity/network, infrastructure and service monitoring, infrastructure maintenance, load balancing, traffic management, capacity and resource planning (i.e., identify the optimal configuration of resources by considering the location-based user requests as well as the performance of the existing hardware resources) [1], [58], [59], [60], resource provisioning (i.e., providing resources, such as memory and computing resources), resource allocation (i.e., assignment of resources to address an incoming request), offline/online centralized/hierarchical/distributed service orchestration, service migration, and task offloading [61], [62], [63], [64], [65], [66], [66], [67], [68], [69], [70]. • The role of AI in the Application/Service Layer: The sheer quantity of IoT data is significant.…”

Dynamic Service Placement in Multi-Access Edge Computing: A Systematic Literature Review

“…The edge servers have limited resources. Designing an independent placement decision for each edge server results in an insufficient utilisation of resources [24], [237], especially when nearby edge servers are under-load or two nearby edge servers have cached similar services. The inefficiency illuminates in cases with a high of services [13].…”

“…By assuming videos correspond services, Yang et al [14] use the Youtube dataset [278] that contains 10,324 videos crawled from the Youtube website on Mar 2nd, 2007. Researchers [15], [81], [237] use a dataset from Google cluster [279] that includes two kinds of data in the log files. The first is the Job-event log files that show the services that users request, and the second is the Task-event log files that show the resources required by the services.…”

QoS‐aware IoT networks and protocols: A comprehensive survey

“…Dependencies between IoT services and QoS of IoT services are also considered during the load balancing process. In their paper, 154 Nezami et al proposed EPOS Fog, a system to perform load balancing during placement of IoT services under cost and QoS constraints. EPOS Fog uses a decentralized multi‐agent approach and collective learning to achieve two optimization objectives.…”