Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues

Tran-Dang, Hoa; Bhardwaj, Sanjay; Rahim, Tariq; Musaddiq, Arslan; Kim, Dong-Seong

doi:10.23919/jcn.2021.000041

Cited by 58 publications

(22 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. , F n 􏼈 􏼉" Output: Energy minimized optimal resource allocation (1) Begin (2) For each dataset "DS" with task "T" and fog nodes "F" //State space (3) Load balancer acquires the input from fog environment "Inp � DS i , WT i , QL i 􏼈 􏼉" (4) Mathematically formulate data size as in equation ( 16) (5) Mathematically formulate waiting time as in equation ( 17) (6) Mathematically formulate queue length as in equation (18) //Action space (7) For each action "A" with the consolidated state "S" (8) If task "T" generated by fog node "F i " is executed locally (9) en "Y i loc � 1" (10) Else "Y i loc � 0" (11) End if (12) If Task "T" generated by fog node "F i " is executed on the host node (13) en "Y F ij � 1" (14) Else "Y F ij � 0" (15) End if (16) If Task "T" generated by fog node "F i " is executed by neighbor (17) en"Y F ijk � 1" (18) Else "Y F ijk � 0" (19) End if //Reward function (20) For each action "A" with the consolidated state "S" and task "T" generated by fog node "F i " (21) Total all the obtained rewards as in equation ( 19) (22) Measure stochastic bellman gradient optimality function as in equation ( 20…”

Section: Experimental Evaluation and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Differential Grey Wolf Load-Balanced Stochastic Bellman Deep Reinforced Resource Allocation in Fog Environment

Nethaji

Chidambaram

2022

Applied Computational Intelligence and Soft Computing

View full text Add to dashboard Cite

Fog computing is becoming a dynamic and sought-after computing prototype for Internet of Things (IoT) application deployments. It works in conjunction with the cloud computing environment. Load balancing, which is employed by IoT applications when deciding, which fog or cloud computing nodes to use, is one of the most critical components for enhancing resource efficiency and avoiding problems like overloading or underloading. However, for IoT applications, ensuring that all CPU nodes are evenly distributed in terms of latency and energy utilization remains a challenge. To solve these issues, this work introduces Differential Grey Wolf (DGW) load balancing with stochastic Bellman deep reinforced resource optimization (DGW-SBDR) in fog situations. A Differential Evolution-based Grey Wolf Optimization algorithm based on load balancing has been developed for optimal resource management. The Grey Wolf Optimization algorithm, which employs differential evolution, assigns jobs to virtual machines based on user demands (VMs). In the event of an overutilized VM pool, a grey wolf optimization strategy based on differential evolution can detect both under and overutilized VMs, allowing for smooth transit between fogs. This step disables a number of virtual machines in order to reduce latency. In a Stochastic Gradient and Deep Reinforcement Learning-based Resource Allocation Model, a stochastic gradient bellman optimality function and Deep Reinforcement Learning are integrated for optimal resource allocation. According to the proposed method, QoS may be supplied to end-users by reducing energy consumption and better managing cache resources utilizing stochastic gradient bellman optimality.

show abstract

Section: Experimental Evaluation and Resultsmentioning

confidence: 99%

“…To ensure service quality, another method called Queuing theorybased cuckoo search was developed in [11]. e fog computing environment's challenges and outstanding issues based on reinforcement learning were discussed in [12].…”

Section: Work That Are Relatedmentioning

confidence: 99%

Differential Grey Wolf Load-Balanced Stochastic Bellman Deep Reinforced Resource Allocation in Fog Environment

Nethaji

Chidambaram

2022

Applied Computational Intelligence and Soft Computing

View full text Add to dashboard Cite

show abstract

“…Second, a two-sided matching is examined in many scenarios as the load balancing, energy consumption criteria are considered in HNs to construct the preference lists. Third, using reinforcement learning algorithms [45] is potential to investigate matching problems where one side may not know their player preference relations a priori. They can only construct their preference list by interacting with players of other side [45].…”

Section: Discussionmentioning

confidence: 99%

“…Third, using reinforcement learning algorithms [45] is potential to investigate matching problems where one side may not know their player preference relations a priori. They can only construct their preference list by interacting with players of other side [45].…”

Section: Discussionmentioning

confidence: 99%

DISCO: Distributed computation offloading framework for fog computing networks

Tran-Dang

Kim

2023

J. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…AI and ML tools provide efficient techniques to analyze and predict the statues of system accurately. Reinforcement learning is a such kind of techniques [82], [83], which can help to build PLs efficiently through online learning mechanism (i.e., exploitation and exploration). Thus, using these in the context of computational offloading enable the system to make dynamic and efficient offloading decisions.…”

Section: F Application Of Ai and Ml-based Techniquesmentioning

confidence: 99%

A Survey on Matching Theory for Distributed Computation Offloading in IoT-Fog-Cloud Systems: Perspectives and Open Issues

Tran-Dang

Kim

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

Fog computing has been widely integrated in the IoT-based systems, creating IoT-Fog-Cloud (IFC) systems to improve the system performances and satisfy the quality of services (QoS) and quality of experience (QoE) requirements for the end users (EUs). This improvement is enabled by computational offloading schemes, which perform the task computation nearby the task generation sources (i.e., IoT devices, EUs) on behalf of remote cloud servers. To realize the benefits of offloading techniques, however, there is a need to incorporate efficient resource allocation frameworks, which can deal effectively with intrinsic properties of computing environment in the IFC systems such as resource heterogeneity of computing devices, various requirements of computation tasks, high task request rates, and so on. While the centralize optimization and non-cooperative game theory based solutions are applicable in a certain number of application scenarios, they fail to be efficient in many of cases, where the global information and control might be unavailable or cost-intensive to achieve it in the large-scale systems. The need of distributed computational offloading algorithms with low computation complexity has motivated a surge of solutions using matching theory. In the present review, we first describe the fundamental concept of this emerging tool enabling the distributed implementation in the computing environment. Then the key solution concepts and algorithmic implementations proposed in the framework of literature are highlighted and discussed. Given the powerful tool of matching theory, its full capability is still unexplored and unexploited in the literature. We thereby discover and discuss existing challenges and corresponding solutions that the matching theory can be applied to resolve them. Furthermore, new problems and open issues for application scenarios of modern IFC systems are also investigated thoroughly. INDEX TERMSIoT-Fog-Cloud systems, Matching theory, Distributed algorithm, Computational offloading. I. INTRODUCTION Practically, the Internet of Things (IoT) has become an integral element for realizing smart practical systems such as smart cities [1], smart grids [2], smart factories [3], smart logistics, and supply chain [4], [5]. The fundamental aspect of IoT-based systems is to connect all devices through the Internet protocol to exchange high volume data and pro-cess them to create smart services and applications [6], [7]. Owing to limited computation resources, network, storage, and energy, IoT devices are inadequate for executing all computational tasks, especially tasks with huge volumes and complex data structures. Cloud computing is an essential solution to this problem because it provides powerful resources to fulfill tasks efficiently [8], [9]. However, cloud

show abstract

Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues

Abstract: Article that has been accepted for inclusion in a future issue of a journal. Content is final as presented, with the exception of pagination.

Cited by 58 publications

References 78 publications

Differential Grey Wolf Load-Balanced Stochastic Bellman Deep Reinforced Resource Allocation in Fog Environment

Differential Grey Wolf Load-Balanced Stochastic Bellman Deep Reinforced Resource Allocation in Fog Environment

DISCO: Distributed computation offloading framework for fog computing networks

A Survey on Matching Theory for Distributed Computation Offloading in IoT-Fog-Cloud Systems: Perspectives and Open Issues

Contact Info

Product

Resources

About