A reinforcement learning-based link quality estimation strategy for RPL and its impact on topology management

Abstract

Over the last few years, standardisation efforts are consolidating the role of the Routing Protocol for Low-Power and Lossy Networks (RPL) as the standard routing protocol for IPv6-based Wireless Sensor Networks (WSNs). Although many core functionalities are well defined, others are left implementation dependent. Among them, the definition of an efficient link-quality estimation (LQE) strategy is of paramount importance, as it influences significantly both the quality of the selected network routes and nodesâ\u80\u99 energy consumption. In this paper, we present RL-Probe, a novel strategy for link quality monitoring in RPL, which accurately measures link quality with minimal overhead and energy waste. To achieve this goal, RL-Probe leverages both synchronous and asynchronous monitoring schemes to maintain up-to-date information on link quality and to promptly react to sudden topology changes, e.g. due to mobility. Our solution relies on a reinforcement learning model to drive the monitoring procedures in order to minimise the overhead caused by active probing operations. The performance of the proposed solution is assessed by means of simulations and real experiments. Results demonstrated that RL-Probe helps in effectively improving packet loss rates, allowing nodes to promptly react to link quality variations as well as to link failures due to node mobility

TAMU‐RPL: Thompson sampling‐based multichannel RPL

“…The RPL protocol has been extensively analyzed and tested in several related works, many of them concerned with RPL's capabilities for convergecast routing and adaptability to interference‐prone and mobile scenarios …”

“…Finally, Ancillotti et al took the first steps towards using a reinforcement learning–based scheme to derive an efficient link quality estimation algorithm that leverages both synchronous and asynchronous measurements. The proposal, called RL‐Probe, carries out asynchronous probing to all the neighbors as soon as certain patterns in the RSSI and ETX measurements are detected.…”

“…The ϵ value chosen in the evaluation was 3%. The experiments by Ancillotti et al did not provide a comprehensive evaluation of the best ϵ value. Moreover, the patterns that are used to start asynchronous probing are statically set for all nodes, which may not be the optimal solution.…”

“…The RPL protocol has been extensively analyzed and tested in several related works, 1,8 many of them concerned with RPL's capabilities for convergecast routing and adaptability to interference-prone and mobile scenarios. 9 A large number of new objective functions have been proposed for improving RPL performance in different scenarios where default ETX-based metrics are combined with more specialized ones that allow the routing tree to be optimized for particular applications. One particularly interesting problem that is addressed by Djedjig et al 10 and Karkazis et al 11 is the trustworthiness of the forwarding nodes.…”

Research on Enhancing RPL for Improved Performance in IOT Networks

“…In native RPL, link quality is estimated using a probing scheme that sends unicast DIO messages to each neighbor which is energy-consuming. To reduce the energy expended during link quality estimation, the authors in [66] used a novel scheme called RPL-Probe based on the multi-armed bandit to measure link quality. This method is reactive to sudden topology changes caused by node mobility.…”