Fog Fragment Cooperation on Bandwidth Management Based on Reinforcement Learning

Abstract

The term big data has emerged in network concepts since the Internet of Things (IoT) made data generation faster through various smart environments. In contrast, bandwidth improvement has been slower; therefore, it has become a bottleneck, creating the need to solve bandwidth constraints. Over time, due to smart environment extensions and the increasing number of IoT devices, the number of fog nodes has increased. In this study, we introduce fog fragment computing in contrast to conventional fog computing. We address bandwidth management using fog nodes and their cooperation to overcome the extra required bandwidth for IoT devices with emergencies and bandwidth limitations. We formulate the decision-making problem of the fog nodes using a reinforcement learning approach and develop a Q-learning algorithm to achieve efficient decisions by forcing the fog nodes to help each other under special conditions. To the best of our knowledge, there has been no research with this objective thus far. Therefore, we compare this study with another scenario that considers a single fog node to show that our new extended method performs considerably better.

Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning

“…In this study, multiple fog nodes of the scenario are set up as a multiagent system. As mentioned in the previous section, the RL approach of [29] employs a fixed-value threshold. In this study, by adding the ability to vary the threshold level (variable threshold), we break the policy into two subpolicies and proceed using HRL.…”

“…The fog node belongs to the only smart fragment of a small environment. This scenario is extended to a vast smart environment with various fog nodes, and each belongs to its distinct fragment [29]. Then, these fog nodes are to cooperate so that the fragment and cooperation conditions are met.…”

“…The fog node is supposed to learn the best selections among devices in ordinary situations to solve this problem and help devices in emergencies by decreasing their transmission rates to the fog node. In the proposed approach of [29,38], the fog node acts as an agent of the RL approach. Although the fog node does not have a trainer or information regarding the distribution parameters of future events, it can successfully gain an optimal policy through RL.…”

“…The vast smart environment of [29] includes several environments called fragments. Each fragment has several IoT devices and a single fog node as the fragment manager responsible for communicating with a higher-level manager (e.g., a cloud server).…”

Next-Generation Internet of Things (IoT): Opportunities, Challenges, and Solutions

“…The outcomes show that a reduced latency and improved reliability can be accomplished by using the proposed approach to create a telemedical healthcare monitoring framework. In [ 4 ], the authors present a fog fragment computing technology (FFCT) solution that is compared with conventional FCT. The paper deals with information transmission using FCT edge nodes and their cooperation to overcome the extra required data transfer capacity for constrained IoT devices.…”