Vision-based Control of a Quadrotor in User Proximity: Mediated vs End-to-End Learning Approaches

Abstract

We consider the task of controlling a quadrotor to hover in front of a freely moving user, using input data from an onboard camera. On this specific task we compare two widespread learning paradigms: a mediated approach, which learns a high-level state from the input and then uses it for deriving control signals; and an end-to-end approach, which skips high-level state estimation altogether. We show that despite their fundamental difference, both approaches yield equivalent performance on this task. We finally qualitatively analyze the behavior of a quadrotor implementing such approaches. VIDEOS, DATASETS, AND CODEVideos, data, and code to reproduce our results are available at: https://github.com/idsia-robotics/ proximity-quadrotor-learning. arXiv:1809.08881v2 [cs.RO]

State-Consistency Loss for Learning Spatial Perception Tasks From Partial Labels

“…Semi-supervised estimation of user pose in a nano-drone: user's head location (x, y, z) and heading (φ) predictions. Compared to the model trained using only the task loss [26] (blue), our approach with ∆t = 0.5s (red) is usually closer to the ground-truth (green). Videos available in supplementary material.…”

“…B. Semi-supervised estimation of user pose in a nano-drone 1) Quantitative analysis: On the held-out testing set, we train the proposed approach using ∆t ∈ {0.5, 1.0, 1.5, 2.0} seconds; we compare the results of these models with two alternatives: one baseline model trained with λ sc = 0 (equivalent to using only labeled data as in [26]); and a model acting as Fig. 5.…”

“…The R 2 metric quantifies the fraction of the variance of the target variable that is explained by the model, and ranges between 0% (for a model that trivially predicts the average of the testing set) and 100% (for an ideal model); this metric does not depend on the unit of measure and is more interpretable (but related to) the MSE. We quantify the improvement of our model over the baseline model, trained using only the task loss [26], both in absolute terms (R 2 points) and, more importantly, as percentage relative to the maximum achievable improvement; this is computed as (model−baseline)/(upper− baseline) and yields 0% for a model that performs as well as the baseline, and 100% for a model that performs as well as the upper-bound.…”

“…7. Semi-supervised estimation of user pose in a nano-drone: R 2 computed for a model trained using only the task loss [26] (baseline), our model trained using task and state-consistency losses using different values of ∆t, and the upper-bound; we report absolute improvements of our approach (∆t = 0.5) vs the baseline, and improvements rescaled such that baseline performance is 0% and upper-bound performance is 100%.…”

“…We consider a Crazyflie 2.1 nano-quadrotor equipped with a front-facing camera X(•) which produces a gray-scale image with size 160 x 96 pixels. The task is a 3-variable regression problem: predicting the relative pose O(•) (frontal displacement x, lateral displacement y, and relative head rotation φ) of a user moving in front of the drone [26].…”