Detection of bodies in maritime rescue operations using unmanned aerial vehicles with multispectral cameras

Abstract

In this study, we use unmanned aerial vehicles equipped with multispectral cameras to search for bodies in maritime rescue operations. A series of flights were performed in open‐water scenarios in the northwest of Spain, using a certified aquatic rescue dummy in dangerous areas and real people when the weather conditions allowed it. The multispectral images were aligned and used to train a convolutional neural network for body detection. An exhaustive evaluation was performed to assess the best combination of spectral channels for this task. Three approaches based on a MobileNet topology were evaluated, using (a) the full image, (b) a sliding window, and (c) a precise localization method. The first method classifies an input image as containing a body or not, the second uses a sliding window to yield a class for each subimage, and the third uses transposed convolutions returning a binary output in which the body pixels are marked. In all cases, the MobileNet architecture was modified by adding custom layers and preprocessing the input to align the multispectral camera channels. Evaluation shows that the proposed methods yield reliable results, obtaining the best classification performance when combining green, red‐edge, and near‐infrared channels. We conclude that the precise localization approach is the most suitable method, obtaining a similar accuracy as the sliding window but achieving a spatial localization close to 1 m. The presented system is about to be implemented for real maritime rescue operations carried out by Babcock Mission Critical Services Spain.

Applications, Evolutions, and Challenges of Drones in Maritime Transport

“…However, the current use of large search and rescue equipment in maritime Synthetic Aperture Radar (SAR) sometimes proves to be inefficient [39,69]. To improve search efficiency, the integration of drones into large rescue equipment has been proposed, allowing for the collaborative modelling of the accident area and the optimisation of search paths for both equipment and rescuers [70].…”

Deep learning with RGB and thermal images onboard a drone for monitoring operations

“…Multispectral solutions that do not use the thermal band yet exhibit impressive results are available. For instance, Gallego et al (2019) detected bodies in a maritime setting utilizing the red edge and near‐infrared spectrum captured using conventional sensors.…”

1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results

“…The open water covers over 70% of our planet and accounts for 80% of international trade [87]. The use of cameras and robotic platforms in this domain is growing fast, and ocean scientists are gathering large amounts of visual data using various sensors with a clear need for robust and reliable methods for quantification, detection, classification and understanding [42,52,53,67,69,76,88,99]. Evidence of this is also reported in recent literature surveys [15,69,103].…”