An Unsupervised Transfer Learning Framework for Visible-Thermal Pedestrian Detection

Abstract

Dual cameras with visible-thermal multispectral pairs provide both visual and thermal appearance, thereby enabling detecting pedestrians around the clock in various conditions and applications, including autonomous driving and intelligent transportation systems. However, due to the greatly varying real-world scenarios, the performance of a detector trained on a source dataset might change dramatically when evaluated on another dataset. A large amount of training data is often necessary to guarantee the detection performance in a new scenario. Typically, human annotators need to conduct the data labeling work, which is time-consuming, labor-intensive and unscalable. To overcome the problem, we propose a novel unsupervised transfer learning framework for multispectral pedestrian detection, which adapts a multispectral pedestrian detector to the target domain based on pseudo training labels. In particular, auxiliary detectors are utilized and different label fusion strategies are introduced according to the estimated environmental illumination level. Intermediate domain images are generated by translating the source images to mimic the target ones, acting as a better starting point for the parameter update of the pedestrian detector. The experimental results on the KAIST and FLIR ADAS datasets demonstrate that the proposed method achieves new state-of-the-art performance without any manual training annotations on the target data.

A Review of Transfer Learning: Advantages, Strategies and Types

“…In conventional supervised learning, a model is trained on labeled instances from the source domain and directly applied to a target domain with a different data distribution, leading to low performance due to domain shift. However, in UTL, the model learns domain-invariant representations, which are features that remain insensitive to domain-specific variations, enabling the model to generalize well to the target domain even without seeing labeled data from that domain during training [17], [18]. UTL finds various applications in real-world scenarios where obtaining labeled instances in the target domain is limited or expensive.…”

DRSNFuse: Deep Residual Shrinkage Network for Infrared and Visible Image Fusion

“…Base part loss.Base parts are defined as areas with similar pixel intensities in infrared and visible images, so the infrared base parts should enjoy high similarity to the visible base parts. The base part loss is computed as (1). We exploit a tanh function to limit the base part loss to the interval of (−1, 1), which can avoid gradient exploding problems in training.…”

“…Infrared images captured by infrared sensors to record the thermal radiations emitted by different objects are widely used in object detection and tracking [ 1 , 2 ]. They are robust to the influence of illumination variation and disguises such as objects in smoke.…”

“…Contributions. The contributions can be summarized as follows: (1) We proposed an end-to-end deep residual shrinkage network (DRSNFuse) for IVIF, which can perform object-aware image fusion. We employed residual shrinkage blocks and a global feature fusion module for encoding, which retains the original textures from infrared and visible images while extracting more semantic features.…”