Far-Infrared Object Segmentation Focus on Transmission of Overall Semantic Information

Abstract

In the vision task of a self-driving system, the use of visible light images to segment an object often loses its functionality at night or in harsh weather. The far-infrared image shows different pixel values according to the thermal radiation quantity of the object itself, so it can be adapted to perform well at night and in harsh weather conditions. However, at the same time, it has insufficient texture features, blurred object boundaries and temperature inversion, which has a great impact on the segmentation task of traditional algorithms. In response to the above problems, this paper proposes a far-infrared object segmentation algorithm using deep learning. In the current popular encoding-decoding structure, multiscale pooling layers are used to obtain receptive fields of different sizes. This is used to solve the effects caused by the blurring of infrared objects. The feature enhancement module is designed for the multireceptive field feature map, which can filter out the most versatile and highly semantic feature channels to reduce the effect of temperature inversion on segmentation. The obtained high semantic feature map is guided into the decoding structure and is fused with the features obtained by the encoder and the decoder. This allows richer information to be obtained between different feature maps. Finally, we also release a new low-resolution far-infrared segmentation dataset. Experiments are performed on three datasets, and the segmentation result of the mIoU(mean Intersection over Union) reaches 70.59%, 30.98% and 60.67%. A large number of experiments confirm the effectiveness and robustness of the network in far-infrared images and verify that the dataset released in this paper has strong reference significance.

Luminescence‐Based Infrared Thermal Sensors: Comprehensive Insights

“…These characteristics are extracted using pedestrian detector based on deep learning an innovative tracing method, time sequenced network based moment detection of long and short term memories, and road surface logical division. Zhang et al [242] urged that far-infrared images exhibit variable pixel values based on the object's thermal radiation, imagery by satellite Himawari Advanced Imager. Reproduced with permission.…”

“…Zhang et al. [ 242 ] urged that far‐infrared images exhibit variable pixel values based on the object's thermal radiation, therefore they can be used at night and in severe weather, but they have insufficient texture information, blurring object boundaries, and temperature inversion, which hinder typical segmentation methods.…”

“…b) Similar dim surroundings while distinct sensing cameras. Reproduced with permission [ 242 ] Copyright 2022, The Institute of Electrical and Electronics Engineers. c) Illustration of encoder‐decoder convolutional neural network.…”