Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine

Abstract

Compared with traditional mine detection methods, UAV-based measures are more suitable for the rapid detection of large areas of scatterable landmines, and a multispectral fusion strategy based on a deep learning model is proposed to facilitate mine detection. Using the UAV-borne multispectral cruise platform, we establish a multispectral dataset of scatterable mines, with mine-spreading areas of the ground vegetation considered. In order to achieve the robust detection of occluded landmines, first, we employ an active learning strategy to refine the labeling of the multispectral dataset. Then, we propose an image fusion architecture driven by detection, in which we use YOLOv5 for the detection part, to improve the detection performance instructively while enhancing the quality of the fused image. Specifically, a simple and lightweight fusion network is designed to sufficiently aggregate texture details and semantic information of the source images and obtain a higher fusion speed. Moreover, we leverage detection loss as well as a joint-training algorithm to allow the semantic information to dynamically flow back into the fusion network. Extensive qualitative and quantitative experiments demonstrate that the detection-driven fusion (DDF) that we propose can effectively increase the recall rate, especially for occluded landmines, and verify the feasibility of multispectral data through reasonable processing.

Deep Learning-Based Real-Time Detection of Surface Landmines Using Optical Imaging

“…The detection of such surface explosive ordnances over large areas is more productive when optical sensors are mounted on UAVs. In [28], the authors propose a solution with two cameras in the VIS and NIR spectrum and the combination by data fusion. The approach is interesting and some preliminary results on the successful detection of small M14 (6 cm diameter) plastic landmines stimulate further developments.…”

Modeling the Effect of Vegetation Coverage on Unmanned Aerial Vehicles-Based Object Detection: A Study in the Minefield Environment

“…Alternatively, spectral-based approaches, such as hyperspectral, multispectral, and thermal infrared imaging (with or without deep learning), show promise for the detection of man-made objects in vegetated environments. Objects such as landmines can be detected based on their distinct spectral signatures that differ from the surrounding environment [20,[53][54][55][56]. These methods are still affected by vegetation occlusion, and therefore quantifying the uncertainty due to vegetation occlusion is an important avenue for future studies for the implementation of such methodologies.…”