SIGS: Synthetic Imagery Generating Software for the Development and Evaluation of Vision-based Sense-And-Avoid Systems

Abstract

Unmanned Aerial Systems (UASs) have recently become a versatile platform for many civilian applications including inspection, surveillance and mapping. Sense-and-Avoid systems are essential for the autonomous safe operation of these systems in non-segregated airspaces. Vision-based Sense-andAvoid systems are preferred to other alternatives as their price, physical dimensions and weight are more suitable for small and medium-sized UASs, but obtaining real flight imagery of potential collision scenarios is hard and dangerous, which complicates the development of Vision-based detection and tracking algorithms. For this purpose, user-friendly software for synthetic imagery generation has been developed, allowing to blend user-defined flight imagery of a simulated aircraft with real flight scenario images to produce realistic images with ground truth annotations. These are extremely useful for the development and benchmarking of Vision-based detection and tracking algorithms at a much lower cost and risk. An image processing algorithm has also been developed for automatic detection of the occlusions caused by certain parts of the UAV which carries the camera. The detected occlusions can later be used by our software to simulate the occlusions due to the UAV that would appear in a real flight with the same camera setup. Additionally this algorithm could be used to mask out pixels which do not contain relevant information of the scene for the visual detection, making the image search process more efficient. Finally an application example of the imagery obtained with our software for the benchmarking of a state-of-art visual tracker is presented.

Study on the Fusion of Oil Painting Art and Digital Media Based on a Visual Sensor

“…Set the number of assembly parts K to control the calculation scale of the assembly curve, and then, determine whether the number of points in the S section is 0 [17] and determine whether to end the cycle, as shown in…”

Part-Based Background-Aware Tracking for UAV With Convolutional Features

“…It is not able to deal with object occlusion effectively. Compared with that, the proposed tracker can update the classifier adaptively, as illustrated in line (2). The tracker can maintain the stability against occlusion situations without the contamination of local appearance models.…”

“…Visual object tracking is a pivotal problem for unmanned aerial vehicle (UAV) applications. In recent years, various tracking methods have been developed to solve challenging problems, such as reconnaissance and surveillance [1], midair monitoring [2], and ship deck landing [3], autonomous chasing [4], infrastructure patrolling [5], pipeline inspection [6], air-to-air refuel [7] and precise landing [8]. Although a plethora of trackers is designed for UAV tracking applications, it is still a tough task to achieve robust tracking, especially in a complex environment.…”

Deep Learning with Semi-Synthetic Training Images for Detection of Non-Cooperative UAVs

“…To optimize the eligibility and variation of the data, artifacts like motion blur or random noise were added to the generated images. The authors of [10] used 3D models of different aircraft from an open source flight simulator on a user defined trajectory in a simulated 3D scene. The generated images have been combined with real video sequences to obtain a labeled data set.…”