E-VAT: An Asymmetric End-to-End Approach to Visual Active Exploration and Tracking

Abstract

Visual active tracking is a growing research topic in robotics due to its key role in applications such as human assistance, disaster recovery, and surveillance. In contrast to passive tracking, active tracking approaches combine vision and control capabilities to detect and actively track the target. Most of the work in this area focuses on ground robots, while the very few contributions on aerial platforms still pose important design constraints that limit their applicability. To overcome these limitations, in this paper we propose D-VAT, a novel end-to-end visual active tracking methodology based on deep reinforcement learning that is tailored to micro aerial vehicle platforms. The D-VAT agent computes the vehicle thrust and angular velocity commands needed to track the target by directly processing monocular camera measurements. We show that the proposed approach allows for precise and collision-free tracking operations, outperforming different state-of-the-art baselines on simulated environments which differ significantly from those encountered during training.

D-VAT: End-to-End Visual Active Tracking for Micro Aerial Vehicles

“…[18] develops an asymmetric dueling training procedure employing an adversarial target that stimulates the development of an effective policy. In [8], the assumption of having the target within the camera FoV at the beginning of the maneuver is removed, so that the agent is able to explore an unknown environment, find the target and track it. All these approaches feature a discrete action space and therefore they cannot explore the full performance envelope of the vehicle.…”

“…For this reason, it is not viable to train the agent using classical RL algorithms, and more advanced solutions based on Deep Neural Network (DNN) approximators must be applied. In particular, we adopt the asymmetric actor-critic formulation [24], [8]. According to this framework [23], we design two different DNN architectures for the actor (A-DNN) and for the critic (C-DNN).…”

“…To evaluate the performance of D-VAT against that of the baselines, we adapted the tracking metrics in [7], [8] to a 3D environment. For convenience, the metrics are defined by expressing the ground-truth position of the target relative to the tracker in a spherical coordinate system, whose axes are aligned with those of the tracker body-fixed frame.…”

“…A viable alternative to overcome these drawbacks is to adopt end-to-end deep reinforcement learning (DRL), which has already shown impressive results in many fields of robotics [4], [5], [6]. Recently, this paradigm has been explored for VAT [7], [8]. Most of the related works focus on ground robots and take advantage of the physical characteristics of these platforms (i.e., low dimensionality of the configuration space and limited number of possible actions) to facilitate the design of VAT policies.…”

Exploring Deep Reinforcement Learning for Robust Target Tracking Using Micro Aerial Vehicles

“…Instead, we employ a DRL strategy that takes advantage of Deep Neural Network (DNN) approximators. In order to develop an effective target tracking policy we adopt an asymmetric actor-critic framework [36], [37]. More specifically, according to this framework [38], we design two DNN architectures: one for the actor (A-DNN) and the other for the critic (C-DNN).…”