STCN-Net: A Novel Multi-Feature Stream Fusion Visibility Estimation Approach

Abstract

Low visibility always leads to serious traffic accidents worldwide, although extensive works are studied to deal with the estimation of visibility in meteorology areas, it is still a tough problem. Deep learning-based visibility estimation methods, it has low accuracy due to lacking "specific features" of the foggy images. Meanwhile, physical modelbased visibility estimation methods are only applicable to some specific scenes due to its high requirements for extra auxiliary parameters. Therefore, This paper proposes a novel end-to-end framework named STCN-Net for visibility estimation, which combined the "engineered features" and "learned features" to achieve higher accuracy. Specifically, a novel 3D multi-feature stream Matrix, named DDT, is designed for visibility estimation, which is consisted of a transmittance matrix, a dark channel matrix, and a depth matrix. Unlike traditional deep learning methods which only use convolutional neural networks (CNN) to deal with the input data or images, our method combines CNN and Transformer to process the input data or images. In STCN-Net, Swin-Transformer(Swin-T) module takes the original image as input while the CNN module takes the DDT matrix as input. Moreover, in order to integrate different feature information from the CNN and Swin-T, we embed a Coordinate Attention (CA) module in STCN-Net. Finally, two visibility datasets: Visibility Image Dataset Ⅰ (VID I) and Visibility Image Dataset Ⅱ (VID II) were constructed for evaluation where VID I is a real scene visibility dataset and VID II is a synthetic visibility dataset. The experimental results show that our method has better performance than classical methods on the two datasets. And compared with the runner-up, it has 2.1% more accuracy in VID I and 0.5% in VID II.

FGS-Net: A Visibility Estimation Method Based on Statistical Feature Stream in Fog Area

“…To visualize the focused regions of FGS-Net in the image when extracting features from the input image. In this paper, we use the grad-cam to make the regions visualization where the network focus on when performing visibility estimation [43]. We draw the results in Figure 14, and in these image, the darker color represents a larger weight value and also means that the network pays more attention to this region.…”

“…Finally, we fuse with features from statistics and the Transformer to estimate the visibility through the FC(fully connected) layer. At last, to verify the effectiveness and superiority of our method, We evaluate our approach on two visibility datasets: VID Ⅰ and VID Ⅱ [3,4,43]. Experimental results show that our method performs better than Giyenko [5], Fatma [6], and various classical methods based on deep learning [7,8,9,10].…”

“…as true positive, false positive, true negative, and false negative.C. DatasetsLiu[21,22,43] constructed two datasets for visibility estimation: VID I and VID II, where VID I is the real-world dataset, VID II is the synthetic dataset. The data in VID I consist of two parts, one part of images come from the cameras of meteorological observatories in Fujian Province, China, and the visibility values are corresponding to the data from instruments of the observatory.…”

Highway Visibility Estimation in Foggy Weather via Multi-Scale Fusion Network

“…We compared the proposed method with several deep-learning-based methods, including two image classification methods (AlexNet [ 21 ] and VGG16 [ 22 ]) and two atmospheric visibility estimation methods (relative CNN-RNN [ 14 ] and STCN-Net [ 23 ]). We re-trained these three deep-learning-based methods on our dataset, where the parameters were set according to the recommendations in the paper.…”

Enhance Low Visibility Image Using Haze-Removal Framework

“…Moreover, research in [40] concentrates on daytime and nighttime conditions. There was also research focused on estimating visibility [41].…”