Drone-Based RGB-Infrared Cross-Modality Vehicle Detection Via Uncertainty-Aware Learning

Sun, Yiming; Cao, Bing; Zhu, Pengfei; Hu, Qinghua

doi:10.1109/tcsvt.2022.3168279

Cited by 164 publications

(58 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section we show results of experiments we have made to evaluate the effectiveness of TSFADet. In section 4.1, we first introduce the DroneVehicle dataset [24], and in section 4.3 we carry out ablation studies for the proposed method on the DroneVehicle dataset. In section 4.4 we compare it with other detection approaches.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Yuan¹,

Wang²,

Wei³

2022

Preprint

View full text Add to dashboard Cite

Integrating multispectral data in object detection, especially visible and infrared images, has received great attention in recent years. Since visible (RGB) and infrared (IR) images can provide complementary information to handle light variations, the paired images are used in many fields, such as multispectral pedestrian detection, RGB-IR crowd counting and RGB-IR salient object detection. Compared with natural RGB-IR images, we find detection in aerial RGB-IR images suffers from cross-modal weakly misalignment problems, which are manifested in the position, size and angle deviations of the same object. In this paper, we mainly address the challenge of cross-modal weakly misalignment in aerial RGB-IR images. Specifically, we firstly explain and analyze the cause of the weakly misalignment problem. Then, we propose a Translation-Scale-Rotation Alignment (TSRA) module to address the problem by calibrating the feature maps from these two modalities. The module predicts the deviation between two modality objects through an alignment process and utilizes Modality-Selection (MS) strategy to improve the performance of alignment. Finally, a two-stream feature alignment detector (TSFADet) based on the TSRA module is constructed for RGB-IR object detection in aerial images. With comprehensive experiments on the public DroneVehicle datasets, we verify that our method reduces the effect of the cross-modal misalignment and achieve robust detection results.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Our experiments were conducted on the DroneVehicle RGB-IR vehicle detection dataset [24]. DroneVehicle is a large-scale drone-based dataset with well-aligned visible/infrared pairs from day to night.…”

Section: Dataset and Evaluation Metricsmentioning

confidence: 99%

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Yuan¹,

Wang²,

Wei³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…DroneVehicle: In order to overcome the low light conditions in UAV visual tasks, DroneVehicle dataset [30] collects 15,532 RGB-Thermal image pairs and 441,642 instances with their resolution at 840×712 pixels. The dataset mainly focuses on the urban field covering roads, parking lots, residential fields, highways and so on.…”

Section: A Object Detectionmentioning

confidence: 99%

“…Website Source Okutama-Action [24] http://okutama-action.org VisDrone2018-2022 [25] https://github.com/VisDrone/VisDrone-Dataset MOR-UAV [26] https://visionintelligence.github.io/Datasets.html CARPK [27] https://paperswithcode.com/dataset/carpk AU-AIR [28] https://bozcani.github.io/auairdataset DroneVehicle [30] https://github.com/VisDrone/DroneVehicle UVSD [29] https://github.com/liuchunsense/UVSD UAVDT [34] https://sites.google.com/view/grli-uavdt/ BIRDSAI [31] https://sites.google.com/view/elizabethbondi/dataset Stanford Drone Dataset [35] https://cvgl.stanford.edu/projects/uav data/ HighD [36] https://www.highd-dataset.com DTB70 [37] https://github.com/flyers/drone-tracking UAV123/20L [38] https://cemse.kaust.edu.sa/ivul/uav123 Anti-UAV [39] https://github.com/ucas-vg/Anti-UAV Small90/112 [40] https://github.com/bczhangbczhang/smallobject UAVDark135 [41] https://vision4robotics.github.io/project/uavdark135/ DarkTrack2021 [42] https://darktrack2021.netlify.app UAVTrack112 [43], [44] https://github.com/vision4robotics/SiamAPN AVSD [45] https://github.com/wyfeng1020/AVSD UAVid [46] https://uavid.nl AeroScapes [47] https://github.com/ishann/aeroscapes ManipalUAVid [48] https://github.com/uverma/ManipalUAVid (49,712) and arbitrary quadrilateral bounding boxes (47,519 small vehicles and 2193 large vehicles). The resolution of these data includes 4000×3000, 5472×3648, and 4056×3040.…”

Section: Datasetsmentioning

confidence: 99%

A Comprehensive Review for Typical Applications Based Upon Unmanned Aerial Vehicle Platform

Han

Liu

Wang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs) have been widely applied in military and civilian fields due to their flexibility and effectiveness. As a vital component of UAVs, the vision system has taken on great significance in different applications (e.g., autonomous landing, traffic surveillance, and disaster rescue) to attract widespread attention in recent years. Therefore, the automatic understanding of visual data collected from these air platforms becomes urgently needed in UAV systems. In this review, we revisit and summarize the recent techniques and developments for several typical UAV applications, including object detection, object tracking, and semantic segmentation. In addition, we also highlight the difficulties and subsequent orientations from different perspectives, which may stimulate future research and applications in the UAV vision era.

show abstract

“…With the continuous development of drone technology [10]- [12], the flexibility and stability of drones are continuously improved, and using drone platforms can well describe the target scenes at different spatial and temporal scales. The traditional Drone view and satellite view image matching technology [13]- [15] is concentrated in the military field, where fixed-wing drones fly at higher flight height and collect images directly below them in real-time to match with their internally stored satellite remote sensing maps to infer the location of the drone.…”

Section: Introductionmentioning

confidence: 99%

SUES-200: A Multi-height Multi-scene Cross-view Image Benchmark Across Drone and Satellite

Zhu¹

2022

Preprint

View full text Add to dashboard Cite

Drone-Based RGB-Infrared Cross-Modality Vehicle Detection Via Uncertainty-Aware Learning

Cited by 164 publications

References 57 publications

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

A Comprehensive Review for Typical Applications Based Upon Unmanned Aerial Vehicle Platform

SUES-200: A Multi-height Multi-scene Cross-view Image Benchmark Across Drone and Satellite

Contact Info

Product

Resources

About