UAV Autonomous Target Search Based on Deep Reinforcement Learning in Complex Disaster Scene

Abstract

In recent years, artificial intelligence has played an increasingly important role in the field of automated control of drones. After AlphaGo used Intensive Learning to defeat the World Go Championship, intensive learning gained widespread attention. However, most of the existing reinforcement learning is applied in games with only two or three moving directions. This paper proves that deep reinforcement learning can be successfully applied to an ancient puzzle game Nokia Snake after further processing. A game with four directions of movement. Through deep intensive learning and training, the Snake (or self-learning Snake) learns to find the target path autonomously, and the average score on the Snake Game exceeds the average score on human level. This kind of Snake algorithm that can find the target path autonomously has broad prospects in the industrial field, such as: UAV oil and gas field inspection, Use drones to search for and rescue injured people after a complex disaster. As we all know, post-disaster relief requires careful staffing and material dispatch. There are many factors that need to be considered in the artificial planning of disaster relief. Therefore, we want to design a drone that can search and rescue personnel and dispatch materials. Current drones are quite mature in terms of automation control, but current drones require manual control. Therefore, the Snake algorithm proposed here to be able to find the target path autonomously is an attempt and key technology in the design of autonomous search and rescue personnel and material dispatching drones. INDEX TERMS Deep reinforcement learning, Markov decision, Monte Carlo, Q-learning. The associate editor coordinating the review of this manuscript and approving it for publication was Zhanyu Ma.

Energy Minimization in D2D-Assisted Cache-Enabled Internet of Things: A Deep Reinforcement Learning Approach

“…P π(s) s,st+1 denotes the transition probability unknown in reality, and π(s) is the action generated under a specific policy. Based on (17) and (18), the cost functions (11), (12) can be rewritten as:…”

“…In order to achieve the ability to learn automatically, we design the updating steps as following [35]: 1) evaluating result: obtain V i,π (s) and V total,π (q) according to (17) and (18) based on the policy π for all status.…”

“…Bellman equations formulate the optimal conditions for our problem but the transition probability is actually unknown in practice. Therefore, we can't compute (17) and (18) directly. In order to obtain an acceptable result, RL algorithms are taken into consideration.…”

“…The authors in [17] investigate the computation offloading problem in blockchain empowered mobile edge computing system, where the deep RL algorithm is applied to the computing offloading decision-making process. Moreover, deep RL is also applied in unmanned aerial vehicle autonomous target searching in a complex disaster scene [18], where the superior ability on dynamic programming of deep RL can be observed. In addition, some well-known deep RL algorithms such as SARSA [19], DQN [20]- [22] are investigated and exploited in practical communication systems.…”

Design and Analysis of the Task Distribution Scheme of Express Center at the End of Modern Logistics

“…The ant colony algorithm [38][39][40][41][42][43] is used to solve the order of shipments in each cluster and to dispatch the shortest path.…”