This paper addresses the local minimum phenomenon, routing path enlargement, and load imbalance problems of geographic routing in wireless sensor networks (WSNs) with holes. These issues may degrade the network lifetime of WSNs since they cause a long detour path and a traffic concentration around the hole boundary. Aiming to solve these problems, in this work, we propose a novel geographic routing protocol for WSNs, namely, Q-learning Inspired Hole bypassing (QIH), which is lightweight and efficient. QIH's conceptual idea is to leverage Q-learning to estimate the distance from a node to the holes. QIH makes routing decisions following the nodes' residual energy, their estimated distance to the holes, and their distance to the destination. We first confirm the effectiveness of QIH by theoretical analysis. Then, we conduct extensive simulations of QIH in comparison to state-of-the-art protocols. The simulation results show that QIH outperforms the other protocols in terms of network lifetime, packet latency, and energy consumption.INDEX TERMS geographic routing, network lifetime, Q-learning, hole bypassing, energy efficiency.
I. INTRODUCTION1 A wireless sensor network (WSN) has sensor nodes deployed 2 over a region of interest with various applications, such as disaster management or agriculture monitoring [1]-[4]. Each 4 sensor node senses its surrounding environment and sends 5 the data via multiple hops to a destination (i.e., the sink).6 Many WSN applications require full sensing coverage, where 7 the death of a single node may cause inefficacious network 8 operations. Moreover, the time from the beginning until the 9 first node's death (i.e., the earliest energy depletion) is de-10 fined as the network lifetime. Hence, one of the most critical 11 problems in WSNs is maximizing the network lifetime. A 12 sensor node typically consumes its energy for computation 13 and transmission tasks. The former energy amount is in-14 significant compared to the latter [5], [6]. Specifically, the 15 energy needed for transmitting a single bit is approximately 16 the same amount for processing a thousand operations [6].Therefore, an energy-efficient routing protocol is essential to 18 conserve energy.
