Improved Deep Q-network for Mobile Charging Sequence Scheduling with the Minimal Mobile Charger Energy Capacity in Wireless Rechargeable Sensor Network

Abstract

Wireless rechargeable sensor network (WRSN) is a new type of wireless sensor networks (WSNs) with rechargeable sensor nodes, which provides a feasible way to overcome the energy constraint prob- lem in WSN. Mobile charging is promising for WRSN by charging sensors using the mobile charger (MC), usually via wireless energy transfer (WET). However, it is challenging to maintain the network working sustainably under the limited capacity of the MC, and most existing works assume that the capacity of MC is infinite or with a very large value, which may be impossible in many real applica- tions. This paper will study the mobile charging sequence scheduling problem to maintain the network working sustainably and minimize the energy capacity of the MC (CSSEC). Specifically, the problem of minimizing the energy capacity of MC is coupled with mobile charging sequence scheduling, and the charging sequence will directly affect the energy cost of MC during a charging tour. Therefore, we propose an improved deep Q-network for CSSEC (IDQN-CSSEC) to address this issue. It employs the MC as the agent to explore the WRSN, and determines a charging policy for MC based on the charging demand of sensors, which selects the next sensor for charging. IDQN-CSSEC uses the Q learning and deep Q-network (DQN) comprehensively, which trains the Q table of the Q learning and the networks of DQN in the early stage of the training, and updates them simultaneously. The action determined by Q learning is selected with a greater probability, which decreases as the iteration step increases. Simu- lation experiments verify that the charging policy determined by IDQN-CSSEC can make the network work sustainably with the minimal energy cost of the MC by comparing it with the baseline methods.