Deep Graph Reinforcement Learning Based Intelligent Traffic Routing Control for Software-Defined Wireless Sensor Networks

Abstract

Software-defined wireless sensor networks (SDWSN), where the data and control planes are decoupled, are more suited to handling big sensor data and effectively monitoring dynamic environments and events. To overcome the limitations of using static routing tables under high traffic intensity, such as network congestion, high packet loss rate, low throughput, etc., it is critical to design intelligent traffic routing control for the SDWSNs. In this paper we propose a deep graph reinforcement learning (DGRL) model-based intelligent traffic control scheme for SDWSNs, which combines graph convolution with deterministic policy gradient. The model fits well for the task of intelligent routing control for the SDWSN, as the process of data forwarding can be regarded as the sampling of continuous action space and the traffic data has strong graph features. The intelligent control policies are made by the SDWSN controller and implemented at the sensor nodes to optimize the data forwarding process. Simulation experiments performed on the Omnet++ platform show that, compared with the existing traffic routing algorithms for SDWSNs, the proposed intelligent routing control method can effectively reduce packet transmission delay, increase packet delivery ratio, and reduce the probability of network congestion.

Software defined wireless sensor load balancing routing for internet of things applications: Review of approaches

“…The work in Ref. [ 107 ] introduced an intelligent routing based on deep graph reinforcement learning for efficient traffic control. An intelligent was incorporated in the controller to extract the network information.…”

“… [ 104 ] Energy-aware routing Reinforcement learning SDWSN Improves network performance in terms of lifetime They overlooked the Quality of services, which may reduce network performance. [ 107 ] Intelligent routing scheme Deep graph reinforcement learning (DGRL) model- SDWSN Improved packet transmission and network congestion. It may face challenges when dealing with highly complex graph-structured data in large-scale network.…”

SSDWSN: A Scalable Software-Defined Wireless Sensor Networks

“…DRL agents continually optimize their neural networks based on the rewards received from their decisions. For example, DGRL [12] combined graph convolution with a deterministic policy gradient through an actor-critic algorithm to intelligently control traffic and reduce the risk of network congestion in SDWSNs. The control policy trained in the controller was implemented in the sensor nodes to optimize the dataforwarding process.…”

Empowering Digital Twin for Future Networks with Graph Neural Networks: Overview, Enabling Technologies, Challenges, and Opportunities

“…The training environment is set up using a Mininet Simulator to simulate a SDN and Ryu Software to simulate controller behavior. Similarly, Huang et al [117] propose a distributed traffic routing control algorithm based on a deep graph reinforcement learning framework that combines the GCN model with a DRL training scheme called Actor-Critic [121]. This framework leverages a GCN model to extract the structural information of the network topology and then generates the next hop policy for each routing request received on individual nodes.…”