SSTN: Self-Supervised Domain Adaptation Thermal Object Detection for Autonomous Driving

Abstract

The sensibility and sensitivity of the environment play a decisive role in the safe and secure operation of autonomous vehicles. This perception of the surrounding is way similar to human visual representation. The human's brain perceives the environment by utilizing different sensory channels and develop a view-invariant representation model. Keeping in this context, different exteroceptive sensors are deployed on the autonomous vehicle for perceiving the environment. The most common exteroceptive sensors are camera, Lidar and radar for autonomous vehicle's perception. Despite being these sensors have illustrated their benefit in the visible spectrum domain yet in the adverse weather conditions, for instance, at night, they have limited operation capability, which may lead to fatal accidents. In this work, we explore thermal object detection to model a view-invariant model representation by employing the self-supervised contrastive learning approach. For this purpose, we have proposed a deep neural network Self Supervised Thermal Network (SSTN) for learning the feature embedding to maximize the information between visible and infrared spectrum domain by contrastive learning, and later employing these learned feature representation for the thermal object detection using multi-scale encoder-decoder transformer network. The proposed method is extensively evaluated on the two publicly available datasets: the FLIR-ADAS dataset and the KAIST Multi-Spectral dataset. The experimental results illustrate the efficacy of the proposed method.

Towards Domain Generalization in Object Detection

“…We consider the classic DG evaluation method to evaluate detectors' generalization ability, where one dataset is selected for test and the others for training for each run. Note that for DG evaluation, knowledge of test distribution is completely inaccessible in the training phase, so that current detection methods designed for domain adaptation (DA) [17,24,34,55] are not applicable. Moreover, most current DG methods are designed for image classification and the adaptation of them in the object detection task is nontrivial [41,60,75].…”

Non-GDANets: Sports small object detection of thermal images with Non-Glodal decoupled Attention

“…Therefore, the attention mechanism is widely used in thermal infrared target detection tasks. For example, Munir and Azam et al [ 29 ] considered that multispectral images can provide complementary information for thermal infrared images and proposed an attention-guided feature fusion method to achieve accurate detection of pedestrians in thermal infrared images. Zhu and Dou et al [ 30 ] proposed a multiscale channel attention image fusion method to achieve the fusion of visible light and thermal infrared images while improving the precision of target detection.…”

Augmentation of Atmospheric Turbulence Effects on Thermal Adapted Object Detection Models

“…However, due to relatively low-scales of IR image sets, detection success on IR imagery is lower when compared to visible band. In order to tackle the data problem in IR imagery, recent studies focus on visible-to-IR domain adaptation and achieve satisfactory results using thermal-RGB pairs and supervised learning [16,27]. Moreover, adversarial-based unsupervised methods are also utilized to thermal domain adaptation [2].…”