End-to-End Detection of a Landing Platform for Offshore UAVs Based on a Multimodal Early Fusion Approach

Abstract

A perception module is a vital component of a modern robotic system. Vision, radar, thermal, and LiDAR are the most common choices of sensors for environmental awareness. Relying on singular sources of information is prone to be affected by specific environmental conditions (e.g., visual cameras are affected by glary or dark environments). Thus, relying on different sensors is an essential step to introduce robustness against various environmental conditions. Hence, a perception system with sensor fusion capabilities produces the desired redundant and reliable awareness critical for real-world systems. This paper proposes a novel early fusion module that is reliable against individual cases of sensor failure when detecting an offshore maritime platform for UAV landing. The model explores the early fusion of a still unexplored combination of visual, infrared, and LiDAR modalities. The contribution is described by suggesting a simple methodology that intends to facilitate the training and inference of a lightweight state-of-the-art object detector. The early fusion based detector achieves solid detection recalls up to 99% for all cases of sensor failure and extreme weather conditions such as glary, dark, and foggy scenarios in fair real-time inference duration below 6 ms.

Energy Efficient Path Planning for 3D Aerial Inspections

“…A 3D LiDAR represents a relevant method to obtain the characteristics of the inspected structure, given its capability of directly obtaining depth, resulting in accurate 3D point clouds of the surrounding environment [10]. These 3D sensors are used in robotics to improve the accuracy in distinct tasks, such as in object detection [11], [12], scene reconstruction and navigation [13], [14] Although a UAV equipped with the required sensors is able to collect the necessary information for an inspection task, its limited flight endurance represents a considerable drawback. To increase the autonomy of the UAVs, it is important to optimize the path that the vehicles will perform as a function of its energy consumption, while assuring that the entire surface of inspection is completely verified.…”

“…When θ = 0 there is no accel- is an increasing function of θ (a xy ̸ = 0). K xy changes according to the ellipsoidal function presented in (12), that depends on θ:…”