Adaptive spatial pixel-level feature fusion network for multispectral pedestrian detection

Fu, Lijun; Gu, Wentao; Ai, Yongbao; Wang, Dong

doi:10.1016/j.infrared.2021.103770

Cited by 17 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is based on the preliminary judgment of an algorithm about information such as the location and category of the specific target; it uses information synthesis to ensure judgment accuracy; however, it relies too much on the previous feature processing results. Pixel-level feature fusion calculates each pixel of each modal image one by one [39]. It is the lowest-level fusion operation, targeting each pixel, which results in considerably complex computation.…”

Section: Dff Modulementioning

confidence: 99%

DMFFNet: Dual-Mode Multi-Scale Feature Fusion-Based Pedestrian Detection Method

Rui

Ouyang

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Most contemporary pedestrian detection algorithms are based on visible light image detection. However, in environments with dim light, small targets, and easily occluded and cluttered backgrounds, single-mode visible light images relying on color, texture, and other features cannot adequately represent the feature information of targets; as a result, a large number of targets are lost and the algorithm performance is not good. To address this problem, we propose a dual-modal multi-scale feature fusion network (DMFFNet). First, we use the MobileNet v3 backbone network to extract the features of dual-modal images as input for the multi-scale fusion attention (MFA) module, combining the idea of multi-scale feature fusion and attention mechanism. Second, we deeply fuse the multi-scale features output by the MFA with the double deep feature fusion (DDFF) module to enhance the semantic and geometric information of the target. Finally, we optimize the loss function to reflect the distance between the predicted box and the real box more realistically as well as to enhance the ability of the network toward predicting difficult samples. We performed multi-directional evaluations on the KAIST dual-light pedestrian dataset and the visible-thermal infrared pedestrian dataset (VTI) in our laboratory through comparative and ablation experiments. The overall MR -2 on the KAIST duallight pedestrian dataset is 9.26%, and the MR -2 in dim light, partial occlusion, and severe occlusion are 5.17%, 23.35%, and 47.31%, respectively. The overall MR -2 on the VIT dual-light pedestrian dataset is 9.26%, and the MR -2 in dim light, partial occlusion, and severe occlusion are 5.17%, 23.35%, and 47.31%, respectively. The results show that the algorithm performs well on pedestrian detection, especially in dim light and when the target was occluded.

show abstract

Section: Dff Modulementioning

confidence: 99%

DMFFNet: Dual-Mode Multi-Scale Feature Fusion-Based Pedestrian Detection Method

Rui

Ouyang

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, it is computationally expensive for our use case and it is not tested for UAVs. Fu et al (2021) introduced a light end-to-end dual-modality multi-scale human detection framework, which can achieve real-time detection speed using an adaptive spatial pixel-level feature fusion (ASPFF) Network. In another study, a unified framework was proposed by Cao et al (2019) which combined the auto-annotation method with a two-stream region proposal network (TS-RPN) detector to learn the semantic features of thermal and optical images to achieve unsupervised learning of multi-spectral features for human detection.…”

Section: Fusion-based Human Detectionmentioning

confidence: 99%

Illumination-aware image fusion for around-the-clock human detection in adverse environments from Unmanned Aerial Vehicle

Golkarnarenji

Martinez‐Alpiste

Wang

et al. 2022

Expert Systems with Applications

View full text Add to dashboard Cite

“…Many scholars have studied and improved aim detection algorithms, for example, Fu, L. et al [ 14 ] An adaptive spatial pixel-level feature fusion network, the ASPFF network, was proposed to detect pedestrian targets on a multiscale feature layer by fusing complementary information from visible and thermal infrared images. Lian, J. et al [ 15 ] proposed a method for detecting small targets in traffic scenes based on attentional feature fusion.…”

Section: Introductionmentioning

confidence: 99%

A Study of an Online Tracking System for Spark Images of Abrasive Belt-Polishing Workpieces

Huang

Zhang

2023

Sensors

View full text Add to dashboard Cite

During the manual grinding of blades, the workers can estimate the material removal rate based on their experiences from observing the characteristics of the grinding sparks, leading to low grinding accuracy and low efficiency and affecting the processing quality of the blades. As an alternative to the recognition of spark images by the human eye, we used the deep learning algorithm YOLO5 to perform target detection on spark images and obtain spark image regions. First the spark images generated during one turbine blade-grinding process were collected, and some of the images were selected as training samples, with the remaining images used as test samples, which were labelled with LabelImg. Afterwards, the selected images were trained with YOLO5 to obtain an optimisation model. In the end, the trained optimisation model was used to predict the images of the test set. The proposed method was able to detect spark image regions quickly and accurately, with an average accuracy of 0.995. YOLO4 was also used to train and predict spark images, and the two methods were compared. Our findings show that YOLO5 is faster and more accurate than the YOLO4 target detection algorithm and can replace manual observation, laying a specific foundation for the automatic segmentation of spark images and the study of the relationship between the material removal rate and spark images at a later stage, which has some practical value.

show abstract

Adaptive spatial pixel-level feature fusion network for multispectral pedestrian detection

Cited by 17 publications

References 31 publications

DMFFNet: Dual-Mode Multi-Scale Feature Fusion-Based Pedestrian Detection Method

DMFFNet: Dual-Mode Multi-Scale Feature Fusion-Based Pedestrian Detection Method

Illumination-aware image fusion for around-the-clock human detection in adverse environments from Unmanned Aerial Vehicle

A Study of an Online Tracking System for Spark Images of Abrasive Belt-Polishing Workpieces

Contact Info

Product

Resources

About