Explainability of Deep Reinforcement Learning Method with Drones

Abstract

Recent advances in artificial intelligence (AI) technology demonstrated that AI algorithms are very powerful as AI models become more complex. As a result, the users and also the engineers who developed the AI algorithms have a hard time explaining how the AI model gives the specific result. This phenomenon is known as "black box" and affects end-users' confidence in these AI systems. In this research, explainability of deep reinforcement learning is investigated for counter-drone systems. To counter a drone, a deep reinforcement learning method such as double deep Q-network with dueling architecture and prioritized experience replay is proposed. In counter-drone systems, catching the target as soon as possible is expected. Otherwise, the target can be gone in a short time. To understand how the agent performs more quickly and accurately, figures representing rewards, drone locations, crash positions, and the distribution of actions are analyzed and compared. For example, the positions of the drones in a successful episode during training can be analyzed by the actions the agent performed and the rewards in this episode. In addition, the actions agent took in episodes are compared with action frequencies during training and it is seen that at the end of the training, the agent selects the dominant actions throughout the training. However, at the beginning of the training, the distribution of actions is not correlated with the actions selected at the end. The results showed that the agent uses different flight paths by using different actions to catch the target drone in different episodes and different models. Finally, the generation of a saliency map is investigated to identify the critical regions in an input image which influences the predictions made by the DQN agent by evaluating the gradients of the model's output with respect to both the image and scalar inputs.