Deep Active Learning from Multispectral Data Through Cross-Modality Prediction Inconsistency

Abstract

Data from multiple sensors provide independent and complementary information, which may improve the robustness and reliability of scene analysis applications. While there exist many large-scale labelled benchmarks acquired by a single sensor, collecting labelled multi-sensor data is more expensive and time-consuming. In this work, we explore the construction of an accurate multispectral (here, visible & thermal cameras) scene analysis system with minimal annotation efforts via an active learning strategy based on the cross-modality prediction inconsistency. Experiments on multispectral datasets and vision tasks demonstrate the effectiveness of our method. In particular, with only 10% of labelled data on KAIST multispectral pedestrian detection dataset, we obtain comparable performance as other fully supervised State-of-the-Art methods.

Multimodal Object Detection of UAV Remote Sensing Based on Joint Representation Optimization and Specific Information Enhancement

“…Modality mAP Faster R-CNN [21] R\I 63.60\75.30% HalfwayFusion [29] R+I 71.17% DALFusion [30] R+I 72.11% CFR [29] R+I 72.39% GAFF [31] R+I 73.80% YOLO-MS [32] R+I 75.20% MFF-YOLOv5 [15] R+I 78.20% UA-CMDet [19] R+I 78.60% FFODNet(ours) R+I 78.30%…”