Shot Type Feasibility in Autonomous UAV Cinematography

Karakostas, Iason; Mademlis, Ioannis; Nikolaidis, Nikos; Pitas, Ioannis

doi:10.1109/icassp.2019.8683014

Cited by 14 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the past years, unmanned aerial vehicles (UAVs) or commonly called drones have proven to be a capable tool in diverse civil domains [1] such as agriculture [2], construction [3], inspection [4], meteorology [5], surveillance [6], photography [7], cinematography [8], consumer entertainment [9], search and rescue [10] and even space exploration [11]. Applications of drones are accordingly manifold and range from exploratory tasks such as visual image mosaicking [12] or simultaneous environment covering tasks [13] to more interactive ones such as fruit harvesting [14] or parcel delivery [15], to name just a few.…”

Section: Introductionmentioning

confidence: 99%

Multidrone Mission Execution With EAMOS: From Text to Mission

Gutmann,

Rinner

2023

IEEE Access

View full text Add to dashboard Cite

Existing software tools for specifying and executing multidrone missions are limited to route planning or tightly coupled to specific drone hardware. We introduce EAMOS (Execution of Aerial Multidrone Missions and Operations Specification Framework), which allows us to specify missions intuitively, text-based, and provides a mission compiler, a mission middle layer, and a distributed drone execution environment. The middle layer wraps the control of individual drone-specific capabilities, such as launch, fly to position, or perform a maneuver, into a public API that transparently utilizes the capabilities of numerous drone platforms. We exploit the Go programming language to implement critical components of the framework and provide an interface for ROS-based drone platforms. EAMOS automates the mission execution on real, virtual, and even hybrid robotic setups involving real and virtual drones. We demonstrate the successful deployment of EAMOS with four missions executed on Pixhawk/PX4-equipped quadcopters and virtual drones simulated with Airsim. We assess the performance of our proposed approach by analyzing the number of nodes and arcs of the mission graphs, which are an essential artifact of our mission compilation, the utilization of ROS service calls during mission execution, and the duration of compilation, deployment, and mission execution. Overall, our experiments showed that our drones correctly behaved during mission execution as expected and specified by their mission, the generated mission artifacts were efficiently manageable, and processing times allowed for a fluent workflow.INDEX TERMS multi-robot missions, software framework, mission execution, drones, ROS, Airsim

show abstract

Section: Introductionmentioning

confidence: 99%

Multidrone Mission Execution With EAMOS: From Text to Mission

Gutmann,

Rinner

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…According to recent UAV shot type taxonomy proposals [3], [4], [5], [6], [7], [8], UAV shots consist in valid combinations This work has received funding from the European Union's Horizon 2020 programme under grant agreement No 951911 (AI4Media).…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning with semi-expert distillation for autonomous UAV cinematography

Sochopoulos,

Mademlis,

Charalampakis

et al. 2023

2023 IEEE International Conference on Multimedia and Expo (ICME)

Self Cite

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs, or drones) have revolutionized modern media production. Being rapidly deployable "flying cameras", they can easily capture aesthetically pleasing aerial footage of static or moving filming targets/subjects. Current approaches rely either on manual UAV/gimbal control by human experts, or on a combination of complex computer vision algorithms and hardware configurations for automating the flight+filming process. This paper explores an efficient Deep Reinforcement Learning (DRL) alternative, which implicitly merges the target detection and path planning steps into a single algorithm. To achieve this, a baseline DRL approach is augmented with a novel policy distillation component, which transfers knowledge from a suitable, semi-expert Model Predictive Control (MPC) controller into the DRL agent. Thus, the latter is able to autonomously execute a specific UAV cinematography task with purely visual input. Unlike the MPC controller, the proposed DRL agent does not need to know the 3D world position of the filming target during inference. Experiments conducted in a photorealistic simulator showcase superior performance and training speed compared to the baseline agent, while surpassing the MPC controller in terms of visual occlusion avoidance.

show abstract

“…Related work A large body of literature exists related to defining shot aesthetics for a videography drone [1]. These works explore parameterizing desired shot qualities and controls [2][3][4] [5], feasibility of dynamic shots [6], and assigning sequences of shots [7] [8]. Other work as focused on algorithmic frameworks for helping directors achieve desirable aesthetic qualities of their shots [9] [10].…”

Section: Introductionmentioning

confidence: 99%

Multi-robot Task Assignment for Aerial Tracking with Viewpoint Constraints

Ray

Pierson

Zhu

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

We address the problem of assigning a team of drones to autonomously capture a set desired shots of a dynamic target in the presence of obstacles. We present a two-stage planning pipeline that generates offline an assignment of drone to shots and locally optimizes online the viewpoint. Given desired shot parameters, the high-level planner uses a visibility heuristic to predict good times for capturing each shot and uses an Integer Linear Program to compute drone assignments. An online Model Predictive Control algorithm uses the assignments as reference to capture the shots. The algorithm is validated in hardware with a pair of drones and a remote controlled car.

show abstract

Shot Type Feasibility in Autonomous UAV Cinematography

Cited by 14 publications

References 16 publications

Multidrone Mission Execution With EAMOS: From Text to Mission

Multidrone Mission Execution With EAMOS: From Text to Mission

Deep Reinforcement Learning with semi-expert distillation for autonomous UAV cinematography

Multi-robot Task Assignment for Aerial Tracking with Viewpoint Constraints

Contact Info

Product

Resources

About