ABiFN: Attention‐based bi‐modal fusion network for object detection at night time

Charan, A. Sai; Jitesh, M.; Chowdhury, Muhammed Tawfiq; Venkataraman, H.

doi:10.1049/el.2020.1952

Cited by 7 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, Cyclic Fuse-and-Refine Network (CFR_3) could reduce the miss rate on the KAIST dataset and improve the mAP on the FLIR benchmark to 72.39%. In November 2020 A. Sai Charan et al attempted to fuse IR and RGB data by giving weight to each sensor using attention mechanism [12]. The attention module is placed in every residual block [13] after the RELU activation function and first batch normalization.…”

Section: Related Workmentioning

confidence: 99%

Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection

Mohsen

Kariminezhad

Mayr

et al. 2022

2022 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

The RGB complementary metal-oxidesemiconductor (CMOS) sensor works within the visible light spectrum. Therefore it is very sensitive to environmental light conditions. On the contrary, a long-wave infrared (LWIR) sensor operating in 8-14 µm spectral band, functions independent of visible light.In this paper, we exploit both visual and thermal perception units for robust object detection purposes. After delicate synchronization and (cross-) labeling of the FLIR [1] dataset, this multi-modal perception data passes through a convolutional neural network (CNN) to detect three critical objects on the road, namely pedestrians, bicycles, and cars. After evaluation of RGB and infrared (thermal and infrared are often used interchangeably) sensors separately, various network structures are compared to fuse the data at the feature level effectively. Our RGB-thermal (RGBT) fusion network, which takes advantage of a novel entropy-block attention module (EBAM), outperforms the state-of-the-art network [2] by 10% with 82.9% mAP. I. INTRODUCTIONA statistical projection of traffic fatalities in the United States for the first half of 2021 shows that an estimated 20,160 people died in motor vehicle traffic crashes. This represents an increase of about 18.4 percent as compared to 17,020 fatalities that were reported in the first half of 2020 [3]. Looking at the fatal accidents of 2019 based on the time, one can see that there are 1,000 more fatal accidents during the night-time compared to the day-time [4]. Given less average traffic during the night-time, the importance of visibility in dark is inevitable.The number of publications on RGB-IR sensor fusion for multi-spectral object detection in the automotive sector has increased within the past two years. However, the lack of data in this research area is still noticeable. There are two main sources of data, namely FLIR thermal dataset [1] and KAIST multi-spectral pedestrian detection benchmark [5], which provide a dataset containing IR and RGB pair images. FLIR mainly provides three classes car, pedestrian, and bicycle, whereas KAIST only contains pedestrians.Nevertheless, the FLIR dataset comes only with IR labels. That introduces the first challenge to researchers. Previously published papers [2], [6] and [7] have made various objections to the dataset. For instance, the usage of different 1 The authors are with the Elektronische Fahrwerksysteme GmbH,

show abstract

Section: Related Workmentioning

confidence: 99%

Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection

Mohsen

Kariminezhad

Mayr

et al. 2022

2022 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

show abstract

“…Zhen et al. design a novel flame detection method with a candidate target area extraction strategy which reaches higher recall rate on the homemade database [10]. An improved faster RCNN fire detection approach based on colour features and image global information is proposed in [13].…”

Section: Introductionmentioning

confidence: 99%

“…With the development of deep learning [2–6], frameworks applying deep convolutional neural networks (CNNs) can detect fires more accurately and efficiently [7–9, 10–13]. Muhammad et al.…”

Section: Introductionmentioning

confidence: 99%

“…Li et al achieve fast and advanced fire detection by using famous detectors, that is, faster RCNN [14], YOLO [15], SSD [16], RFCN [17], to remedy the low accuracy of conventional methods [9]. Zhen et al design a novel flame detection method with a candidate target area extraction strategy which reaches higher recall rate on the homemade database [10]. An improved faster RCNN fire detection approach based on colour features and image global information is proposed in [13].…”

mentioning

confidence: 99%

“…With the development of deep learning [2][3][4][5][6], frameworks applying deep convolutional neural networks (CNNs) can detect fires more accurately and efficiently [7][8][9][10][11][12][13]. Muhammad et al propose a computationally efficient CNN structure to localize, and understand the semantic of fire scenes [8].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Dynamic attention network for flame detection

Fang

Zhang

et al. 2023

Electronics Letters

View full text Add to dashboard Cite

Recently, flame detection has attracted great attention. However, existing methods have the issues of low detection rates, high false alarm rates, and lack of smoke anti-interference ability. In this letter, a novel dynamic attention-based network (DANet) is proposed for autonomous flame detection in various scenarios. To mitigate the disturbance of smoke in images, a dynamic attention strategy is proposed to discover the potential features among scale-awareness and spatial-awareness. Then, based on dynamic attention module, a decoupled detection head is presented, which can predict category, regression, and object score independently to boost the performance. A self-contained challenging flame dataset, which is multi-scene, multiscale, and multi-interference informative is constructed to evaluate the proposed model and organize the experiments. Extensive ablation and comparison studies on self-labelled dataset reveal the effectiveness of the proposed dynamic attention-based network.

show abstract

BMFN3D: Bidirectional multilayer fusion network for indoor 3D object detection

Cheng

Zhang

2022

Electronics Letters

View full text Add to dashboard Cite

Among various indoor 3D object detection methods, one of which is based on 3D convolution. However, in current 3D convolutional method for indoor 3D object detection, feature fusion is only implemented in a single direction and adjacent layer, which leads to the feature loss during feature propagation and weakens the detection performance. To solve the above problem, a bidirectional multilayer fusion network (BMFN) is proposed to enrich features for prediction. Furthermore, we propose a new network based on BMFN for 3D object detection, BMFN3D. In BMFN3D, the channel and spatial attention mechanisms are introduced into the backbone and head of detection network, so as to enhance the selection of effective feature information. Finally, a new calculation method for regression loss is proposed with the consideration of the 3D spatial information. We validate our method with two indoor datasets of SUN RGB-D and ScanNet, and achieve state-of-the-art results.

show abstract

ABiFN: Attention‐based bi‐modal fusion network for object detection at night time

Cited by 7 publications

References 23 publications

Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection

Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection

Dynamic attention network for flame detection

BMFN3D: Bidirectional multilayer fusion network for indoor 3D object detection

Contact Info

Product

Resources

About