Enhancing UAV Detection in Surveillance Camera Videos through Spatiotemporal Information and Optical Flow

Abstract

The growing intelligence and prevalence of drones have led to an increase in their disorderly and illicit usage, posing substantial risks to aviation and public safety. This paper focuses on addressing the issue of drone detection through surveillance cameras. Drone targets in images possess distinctive characteristics, including small size, weak energy, low contrast, and limited and varying features, rendering precise detection a challenging task. To overcome these challenges, we propose a novel detection method that extends the input of YOLOv5s to a continuous sequence of images and inter-frame optical flow, emulating the visual mechanisms employed by humans. By incorporating the image sequence as input, our model can leverage both temporal and spatial information, extracting more features of small and weak targets through the integration of spatiotemporal data. This integration augments the accuracy and robustness of drone detection. Furthermore, the inclusion of optical flow enables the model to directly perceive the motion information of drone targets across consecutive frames, enhancing its ability to extract and utilize features from dynamic objects. Comparative experiments demonstrate that our proposed method of extended input significantly enhances the network’s capability to detect small moving targets, showcasing competitive performance in terms of accuracy and speed. Specifically, our method achieves a final average precision of 86.87%, representing a noteworthy 11.49% improvement over the baseline, and the speed remains above 30 frames per second. Additionally, our approach is adaptable to other detection models with different backbones, providing valuable insights for domains such as Urban Air Mobility and autonomous driving.

Risks of Drone Use in Light of Literature Studies

“…Detecting drones in surveillance footage is challenging due to their small size, low contrast, and bird similarity. To solve this problem, researchers propose using the following drone detection techniques: Deep machine learning [ 251 , 252 ]; Deep convolutional neural network (DC-CNN) (DC-CNN) [ 253 ]; Spatiotemporal information and optical flow [ 254 ]; Radio frequency (RF) [ 255 , 256 , 257 ]; Sensors that measure the sound emitted by the UAV [ 258 ]; The transformer network [ 259 ]; The “fisheye” camera system [ 260 ]. …”

CMCA-YOLO: A Study on a Real-Time Object Detection Model for Parking Lot Surveillance Imagery

“…Although these methods have made some progress in enhancing the model's generalizability and dealing with complex monitoring environments, specific challenges unique to parking lot monitoring, such as extreme changes in light intensity and high-density target occlusion, still need to be addressed. Therefore, to further improve the recognition accuracy and real-time performance of parking lot monitoring systems, some studies have begun exploring new avenues, such as [50] introducing continuous image sequences and frame-to-frame optical flow processing methods to simulate human visual mechanisms and [42,51] aiming to enhance the detection capability for small moving targets by improving model structures and loss functions. These innovative methods have significantly improved the performance of monitoring models under specific conditions, but their universality and robustness in actual parking lot monitoring applications, especially in dealing with multi-target occlusion and extreme weather conditions in image capture, remain key issues for current research to explore in depth.…”

An Efficient Adjacent Frame Fusion Mechanism for Airborne Visual Object Detection

“…Second, this method cannot identify drone targets larger than 32 × 32 pixels because it can only extract the edge motion features of the target, which can lead to the incorrect positioning of the target. In addition to the aforementioned approach, several researchers [15][16][17] have proposed utilizing multi-frame information to enhance model performance. However, these methods suffer from issues such as excessive computational steps or a substantial increase in calculations.…”