Visual-Based Person Detection for Search-and-Rescue with UAS: Humans vs. Machine Learning Algorithm

Abstract

Unmanned Aircraft Systems (UASs) have been recognized as an important resource in search-and-rescue (SAR) missions and, as such, have been used by the Croatian Mountain Search and Rescue (CMRS) service for over seven years. The UAS scans and photographs the terrain. The high-resolution images are afterwards analyzed by SAR members to detect missing persons or to find some usable trace. It is a drawn out, tiresome process prone to human error. To facilitate and speed up mission image processing and increase detection accuracy, we have developed several image-processing algorithms. The latest are convolutional neural network (CNN)-based. CNNs were trained on a specially developed image database, named HERIDAL. Although these algorithms achieve excellent recall, the efficiency of the algorithm in actual SAR missions and its comparison with expert detection must be investigated. A series of mission simulations are planned and recorded for this purpose. They are processed and labelled by a developed algorithm. A web application was developed by which experts analyzed raw and processed mission images. The algorithm achieved better recall compared to an expert, but the experts achieved better accuracy when they analyzed images that were already processed and labelled.

Rethinking Drone-Based Search and Rescue with Aerial Person Detection

“…According to Pyrrö et al [35], the average human achieves roughly the following metrics in SAR-APD: 59% PRC, 68% RCL and 33 s ATI, with less than around 200 images to inspect. This is a fraction of the average, real-world number [12]. Moreover, the human performance is demonstrated to drop with the number of images inspected due to accumulated fatigue [35].…”

“…Search and rescue (SAR) missions have been carried out for centuries to aid those who are lost or in distress, typically in some remote areas, such as wilderness. With recent advances in technology, small unmanned aerial vehicles (UAVs) or drones have been used during SAR missions for years in many countries [3,10,12,30,38]. The reason is that these drones enable rapid aerial photographing of large areas with potentially difficult-to-reach terrain, which improves the safety of SAR personnel during the search operation.…”

VisionICE: Air–Ground Integrated Intelligent Cognition Visual Enhancement System Based on a UAV

“…Yang et al [2] utilized unmanned aerial vehicles and unmanned surface vehicles to collaborate for maritime search and rescue, and used reinforcement learning (RL) to achieve path planning. Gotovac et al [5] utilized drones to pre-acquire aerial images, and then used convolutional neural networks to improve the efficiency and reliability of search and rescue. The problem with this method is that it cannot be detected in real time.…”

“…In recent years, computer vision technology has made breakthroughs with the support of big data processing and high-performance cloud computing. The integration of computer vision technology and UAV technology has effectively addressed UAV surveillance more significantly [4,5] and is a powerful tool for achieving situational awareness, target indication [6], and ground target tracking [7,8]. Therefore, combining post-disaster search and rescue and intelligent UAVs gives the searchers both ground and air perspectives.…”

SaRNet: A Dataset for Deep Learning Assisted Search and Rescue with Satellite Imagery

“…The use of deep learning to assist in SaR operations has been explored a number of times [2], [4], [13] but not as far as we can tell with satellite imagery. The authors of [13] discuss how the human eye has incredible power to use context to discern true from false targets but is slow to scan images and can quickly become fatigued.…”