Integration of the 3D Environment for UAV Onboard Visual Object Tracking

Abstract

Single visual object tracking from an unmanned aerial vehicle (UAV) poses fundamental challenges such as object occlusion, small-scale objects, background clutter, and abrupt camera motion. To tackle these difficulties, we propose to integrate the 3D structure of the observed scene into a detection-by-tracking algorithm. We introduce a pipeline that combines a model-free visual object tracker, a sparse 3D reconstruction, and a state estimator. The 3D reconstruction of the scene is computed with an image-based Structure-from-Motion (SfM) component that enables us to leverage a state estimator in the corresponding 3D scene during tracking. By representing the position of the target in 3D space rather than in image space, we stabilize the tracking during ego-motion and improve the handling of occlusions, background clutter, and small-scale objects. We evaluated our approach on prototypical image sequences, captured from a UAV with low-altitude oblique views. For this purpose, we adapted an existing dataset for visual object tracking and reconstructed the observed scene in 3D. The experimental results demonstrate that the proposed approach outperforms methods using plain visual cues as well as approaches leveraging image-space-based state estimations. We believe that our approach can be beneficial for trafficmonitoring, video surveillance, and navigation.

A Multilevel Architecture for Autonomous UAVs

“…The vision thread also receives some additional data, such as the attitude, relative and absolute position given by the t-265 stereoscopic camera and by the GNSS (if available). Inside this thread, various computer vision algorithms can be implemented, such as environmental marker detection [21,22] or object recognition [23,24]; 3. Inertial and GNSS thread.…”

“…Finally, Byte 24 is the closing byte of the frame. Byte [24] Footer Figure 4. This figure shows the frame architecture according to the SBUS standard, which unlike the two previous standards, is a digital protocol.…”

Lightweight air-to-air unmanned aerial vehicle target detection model

“…Unmanned Aerial Vehicles (UAVs), commonly known as drones, are autonomous aircraft capable of carrying various payloads, executing multiple tasks, and being reusable. Their unique features, such as cost-effectiveness, minimal losses, zero human casualties, high maneuverability, stealth capabilities, and adaptability, have significantly contributed to their widespread use in military, civilian, and scientific research fields 1 – 4 , thus fostering the growth of the drone industry.…”

Geo-Referenced Mapping through an Anti-Collision Radar Aboard an Unmanned Aerial System

“…Currently, vision systems are often used with this aim [4][5][6]. However, their maximum detectable distance (usually lower than tens of meters) is a serious drawback [7], which does not make them the optimal solution for mapping outdoor environments.…”