Computer Vision for Autonomous UAV Flight Safety: An Overview and a Vision-based Safe Landing Pipeline Example

Abstract

Recent years have seen an unprecedented spread of Unmanned Aerial Vehicles (UAVs, or “drones”), which are highly useful for both civilian and military applications. Flight safety is a crucial issue in UAV navigation, having to ensure accurate compliance with recently legislated rules and regulations. The emerging use of autonomous drones and UAV swarms raises additional issues, making it necessary to transfuse safety- and regulations-awareness to relevant algorithms and architectures. Computer vision plays a pivotal role in such autonomous functionalities. Although the main aspects of autonomous UAV technologies (e.g., path planning, navigation control, landing control, mapping and localization, target detection/tracking) are already mature and well-covered, ensuring safe flying in the vicinity of crowds, avoidance of passing over persons, or guaranteed emergency landing capabilities in case of malfunctions, are generally treated as an afterthought when designing autonomous UAV platforms for unstructured environments. This fact is reflected in the fragmentary coverage of the above issues in current literature. This overview attempts to remedy this situation, from the point of view of computer vision. It examines the field from multiple aspects, including regulations across the world and relevant current technologies. Finally, since very few attempts have been made so far towards a complete UAV safety flight and landing pipeline, an example computer vision-based UAV flight safety pipeline is introduced, taking into account all issues present in current autonomous drones. The content is relevant to any kind of autonomous drone flight (e.g., for movie/TV production, news-gathering, search and rescue, surveillance, inspection, mapping, wildlife monitoring, crowd monitoring/management), making this a topic of broad interest.

Neural Attention-Driven Non-Maximum Suppression for Person Detection

“…• (4-7) the normalized width/height/area/aspect-ratio of the j th ROI. • (8)(9)(10)(11) the width/height/area/aspect-ratio differences (e.g., wj wi ) between the j th and the i th ROI. 1 Horizontal and vertical distances are signed distances.…”

“…The proposed method is highly applicable to the person/pedestrian detection task, where most NMS algorithms face difficulties in identifying individuals within a crowd, due to various levels of occlusions. The majority of existing NMS methods are oriented towards fast execution, but person detection in human crowds requires a high degree of accuracy; this is critical for ensuring human safety in domains such as autonomous systems [8] [9] [10] [11] [12] [13] [14] [15]. Moreover, the visual appearance representation approach adopted by Seq2Seq-NMS, i.e., FMoD descriptors computed on edge maps of cropped candidate ROIs, is most accurate in cases where the visible silhouette of the target object class remains approximately identical in shape across the training and test images.…”

Trajectory Tracking Control Method for Flexible Robot of Construction Machinery Based on Computer Vision

“…At present, 3D Cartesian coordinate system has been used abroad for trajectory planning. Because X-ray scanning technology, geometric constraint control theory based on kinematic equations and two-dimensional point tracking algorithm can achieve high tracking accuracy of industrial robots and can track objects of arbitrary shape in the straight circumferential direction [11][12].…”

An Extended Model for the UAVs-Assisted Multiperiodic Crowd Tracking Problem

“…Te study of crowds dynamicity has been taken as an important issue in the UAVs management [20,22,29,37]. Designing an efcient crowd monitoring and analysis (CMA) system is a challenging task to ensure security and safety in crowded events [38][39][40][41][42]. Indeed, a strategic monitoring and tracking of crowds could give an opportunity for early decisions to prevent fatal accidents due to abnormal behavior.…”