Real-Time Drone Detection and Tracking With Visible, Thermal and Acoustic Sensors

Abstract

This paper explores the process of designing an automatic multi-sensor drone detection system. Besides the common video and audio sensors, the system also includes a thermal infrared camera, which is shown to be a feasible solution to the drone detection task. Even with slightly lower resolution, the performance is just as good as a camera in visible range. The detector performance as a function of the sensor-to-target distance is also investigated. In addition, using sensor fusion, the system is made more robust than the individual sensors, helping to reduce false detections. To counteract the lack of public datasets, a novel video dataset containing 650 annotated infrared and visible videos of drones, birds, airplanes and helicopters is also presented. The database is complemented with an audio dataset of the classes drones, helicopters and background noise.

Reinforcement Learning for Pan-Tilt-Zoom Camera Control, with Focus on Drone Tracking

“…For our PTZ camera, the őeld of regard (FOR) and the őeld of view (FOV) do not coincide. We set the limits of the camera movement to be: tilt [30,330]°, and zoom [20,200] mm. Yaw is unlimited.…”

“…This can be either a person őlming a drone freehand or a human operator controlling a PTZ camera. • Automated PTZ camera detection [19,20]: a fully automated control system for controlling the PTZ cameras to detect any ŕying drones. There are advantages and disadvantages to each of the methods.…”

“…Their architecture consists of detecting ŕying objects in the video feed, calculating the trajectory, controlling the PTZ, and classifying the objects. Similarly, Svanstrom et al [20] use a pan-tilt platform to control multiple sensors: a video camera, an infrared camera, a ősheye lens, and a microphone. By fusing the data from the sensors, they are able to correctly classify objects.…”

On Onboard LiDAR-based Flying Object Detection

“…Surveys of state-of-the-art C-UAS systems such as [1], [2] rarely consider solutions based on LiDAR detection, which are relatively sparse when compared to other detection methods such as RADAR [13], [14], visual [6], acoustic [15] or even using mixed sensors [16]. Most of the existing solutions for detecting UAVs are ground-based and rely on the assumption that the sensor is stationary, which also applies to most LiDAR-based methods such as [17] or [18].…”

“…2) Point cloud update: The newPointCloud() routine processes incoming LiDAR scans P. These are used to update the buffer B (lines 9-12) and the set of active tracks T (lines [13][14][15][16][17][18]. We assume that some tracked objects can disappear (e.g.…”