Visual Object Detection For Autonomous UAV Cinematography

Abstract

The popularization of commercial, battery-powered, camera-equipped, Vertical Take-off and Landing (VTOL) Unmanned Aerial Vehicles (UAVs) during the past decade, has significantly affected aerial video capturing operations in varying domains. UAVs are affordable, agile and flexible, having the ability to access otherwise inaccessible spots. However, their limited resources burden computation cinematography techniques on operating with high accuracy and real-time speed on such devices. State-of-the-art object detectors and feature extractors are, thus, studied in this work, aiming to find a trade-off between performance and speed that will allow UAV exploitation for intelligent cinematography purposes. Experimental evaluation on three newly introduced datasets of rowing boats, cyclists and parkour athletes is performed and evidence is provided that even limited-resource autonomous UAVs can indeed be used for cinematography applications.

A multiple-UAV architecture for autonomous media production

“…High-performance, real-time object detection and 2D visual tracking for embedded devices is achievable today with deep Convolutional Neural Networks [27] [47] [50] and neural correlation-based trackers [26] [53], respectively, that are typically executed in parallel on GP-GPUs. The algorithms employed in this module are detailed in [40] [39] [41] [44]; its most important building block is a lightweight deep neural object detector integrated with a much faster 2D visual object tracker, so that the first one automatically initializes the latter one to the target ROI detected the closest to the image center, while subsequently re-initializing it periodically on a needto-run basis (i.e., when the module decides that the tracker has drifted and has lost track of the target). The involved lightweight deep CNNs were pretrained in a supervised manner on a regular desktop PC, using domain-specific, manually annotated datasets, and then deployed on the UAV computational hardware at inference mode only.…”

Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction

“…Lastly, a one-hot-encoding vi of the estimated view class v i ∈ R 3 is used, where v Back and v Ambiguous are combined into one common class. Then W, H, W × H, d (1,11) and vi are concatenated into a 7-dimensional input vector for the distance estimation. The area W × H ∈ R is a vital feature for the distance estimation, thus we represent it explicitly as input instead of learning it from H and W implicitly.…”

“…As drones, or Unmanned Ariel Vehicles (UAVs), have become increasingly more accessible, the interest in integrating them into different aspects of our lives is continuously growing. As such, drones are not only used for photography and filming [1], [2], or shipping and delivery [3], but they are also utilized for navigating visually impaired persons [4]. For this task, assisting drones are often operated via a remote control [5].…”