Using a qualitative and quantitative validation methodology to evaluate a drone detection system

Abstract

Now that the use of drones is becoming more common, the need to regulate the access to airspace for these systems is becoming more pressing. A necessary tool in order to do this is a means of detecting drones. Numerous parties have started the development of such drone detection systems. A big problem with these systems is that the evaluation of the performance of drone detection systems is a difficult operation that requires the careful consideration of all technical and non-technical aspects of the system under test. Indeed, weather conditions and small variations in the appearance of the targets can have a huge difference on the performance of the systems. In order to provide a fair evaluation, it is therefore paramount that a validation procedure that finds a compromise between the requirements of end users (who want tests to be performed in operational conditions) and platform developers (who want statistically relevant tests) is followed. Therefore, we propose in this article a qualitative and quantitative validation methodology for drone detection systems. The proposed validation methodology seeks to find this compromise between operationally relevant benchmarking (by providing qualitative benchmarking under varying environmental conditions) and statistically relevant evaluation (by providing quantitative score sheets under strictly described conditions).

Wavelet transform analytics for RF-based UAV detection and identification system using machine learning

“…There is no automatic system in place for law enforcement agencies to detect airspace contravention caused by UAVs. Solutions are still under development or at infant stages in development [8][9][10]. Having this solution will enable law enforcement to tackle UAV crimes.…”

Analysis of User Experience Online Mentoring Platform with User-Centered Design Approach at Giza Design Lab

“…The lack of planning can be one of the critical triggering factors to the use of User-Centred Design, which should be focused and integrated into all phases of the system life cycle [8] [9]. Testing and operational validation during each of the stages of the design process (requirements identification, proof of concept development, prototype development, final product development) is absolutely necessary, as it is often very difficult for the designers of a product to understand intuitively [10] [11].…”

A Method of Fast and Simultaneous Calibration of Many Mobile FMCW Radars Operating in a Network Anti-Drone System

“…For example, the report by Bard College's Center for the Study of the Drone [3] found that 327 incidents between December 2013 and September 2015 (USA only) posed a "proximity danger" where an unmanned aircraft got within 500 feet (152.4 m) of a plane, helicopter, or other manned aircraft (involved multiengine jet aircraft). The above mentioned, as well as similar accidents, have caused many institutions to Many scientists and engineers involved in the development of anti-drone systems agree that a multi-sensor system could provide a higher probability of drone detection than a single sensor [6][7][8][9][10][11][12]. Experiments also suggest that one of the most efficient detectors of UAV is a frequency modulated continuous wave (FMCW) radar [6][7][8][9][10][11][12].…”

“…The above mentioned, as well as similar accidents, have caused many institutions to Many scientists and engineers involved in the development of anti-drone systems agree that a multi-sensor system could provide a higher probability of drone detection than a single sensor [6][7][8][9][10][11][12]. Experiments also suggest that one of the most efficient detectors of UAV is a frequency modulated continuous wave (FMCW) radar [6][7][8][9][10][11][12]. It has a big detection range, transmits low power [13], and is independent of electromagnetic signals and sounds emitted by drones (UAV operating in a fully autonomous mode and gliding can be detected, too).…”