Deep Camera-Radar Fusion with Attention Framework for Autonomous Vehicle Vision in Foggy Weather Conditions

Ogunrinde, Isaac Oluwadunsin; Bernadin, Shonda

doi:10.20944/preprints202305.2180.v1

Cited by 6 publications

(6 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 2023, Ogunrinde, I.O. et al, [16] design a CR-YOLOnet network using A multi-sensor fusion network based on YOLOv5 that combines radar object recognition with camera image bounding box. To train and test our multisensor fusion network using CARLA simulator clear and multifog weather datasets.…”

Section: Literature Surveymentioning

confidence: 99%

Deep-APT: Deep Learning based Efficient Accident Prevention Technique in Fogged Environment

Lokesh

Ahilan³

2023

Preprint

View full text Add to dashboard Cite

Image defogging is an innovative strategy for recovering images in foggy environments that has gotten a lot of attention in recent years because of its use in surveillance systems. The standard defogging algorithm, on the other hand, has difficulty merging the depth of picture detail and the colour of the picture. In this paper, a novel Accident Prevention Technique (Deep-APT) has been proposed to effectively restore fog-free images and prevent accidents using FasterRCNN network. Initially, a dashboard camera monitors the road ahead of the vehicle and collects video. This video sequence is converted to frames. The transformed images are pre-processed using an Adaptive dual threshold Tetrolet transform that preprocess foggy images to fog-free images it is used to remove noise in the input image. Based on the defogged image, use FasterRCNN technology to detect objects in front of the car. The Deep-APT method has been simulated using MATLAB. The experimental result shows the proposed Deep-APT yields an overall accuracy is 99.52%. As compared to existing techniques, the proposed FasterRCNN network shows better results in terms of precision, F1 score, accuracy, and recall. Using DAWN dataset, the MSE, SSIM and PSNR values for the proposed method are 0.12, 0.65 and 0.12. The Deep-APT network improves the overall accuracy of 15.43%, and 4.72% better than CR-YOLnet, and RDL respectively.

show abstract

Section: Literature Surveymentioning

confidence: 99%

Deep-APT: Deep Learning based Efficient Accident Prevention Technique in Fogged Environment

Lokesh

Ahilan³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…AlexNet, suggested by Krizhvsky et al [37], was the first convolutional network used for image feature extraction, ushering in the current era of deep feature extraction. In our previous work [31], we did a comprehensive review of camera-only along with camera and radar fusion-based object detection methods. Some of the camera-only approaches include SSD proposed by Liu et al [38], YOLO proposed by Redmon et al [39], and its derivatives [40][41][42][43][44], RCNN proposed by Girshick et al [45], and its derivatives [46][47][48].…”

Section: Object Detectionmentioning

confidence: 99%

“…Because of the tradeoff between detection speed and accuracy, existing methods have a very limited range of use in foggy weather conditions. Recently, we proposed a deep learning-based radar and camera fusion (CR-YOLOnet) [31]based on YOLOv5 [44] for object detection in foggy weather conditions. In [31], we gave a comprehensive overview of YOLOv5.…”

Section: Object Detectionmentioning

confidence: 99%

“…Recently, we proposed a deep learning-based radar and camera fusion (CR-YOLOnet) [31]based on YOLOv5 [44] for object detection in foggy weather conditions. In [31], we gave a comprehensive overview of YOLOv5. When detecting small and distant objects, our small CR-YOLOnet model achieved a balance between accuracy and speed and performed better than YOLOv5.…”

Section: Object Detectionmentioning

confidence: 99%

“…We make the following contributions to achieve improved tracking speed and tracking accuracy under foggy weather conditions:  Instead of a single sensor modal system (video sequence only) used for the detection phase in [9,10], we employed our deep camera-radar fusion network (CR-YOLOnet model) [31] for faster and more accurate object detection in the detection phase of our improved deep learning-based MOT in Figure 2. Our CR-YOLOnet model reached an accuracy (mAP at 0.50) of 0.849 and a speed of 69 fps for small and distant object detection under medium and heavy fog conditions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network

Ogunrinde¹,

Bernadin²

2023

Preprint

View full text Add to dashboard Cite

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster-tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase, and a simple neural network for deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to im-prove tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and cost. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, the speed increased by 37.56%, and identity switches decreased by 46.81%.

show abstract

Integrating Knowledge Graphs into Autonomous Vehicle Technologies: A Survey of Current State and Future Directions

Htun,

Fukuda

2024

Information

View full text Add to dashboard Cite

Autonomous vehicles (AVs) represent a transformative innovation in transportation, promising enhanced safety, efficiency, and sustainability. Despite these promises, achieving robustness, reliability, and adherence to ethical standards in AV systems remains challenging due to the complexity of integrating diverse technologies. This survey reviews literature from 2017 to 2023, analyzing over 90 papers to explore the integration of knowledge graphs (KGs) into AV technologies. Our findings indicate that KGs significantly enhance AV systems by providing structured semantic understanding, improving real-time decision-making, and ensuring compliance with regulatory standards. The paper identifies that while KGs contribute to better environmental perception and contextual reasoning, challenges remain in their seamless integration with existing systems and in maintaining processing speed. We also address the ethical dimensions of AV decision-making, advocating for frameworks that prioritize safety and transparency. This review underscores the potential of KGs to address critical challenges in AV technologies, offering a hopeful and optimistic outlook for the development of robust, reliable, and socially responsible autonomous transportation solutions.

show abstract

Deep Camera-Radar Fusion with Attention Framework for Autonomous Vehicle Vision in Foggy Weather Conditions

Cited by 6 publications

References 67 publications

Deep-APT: Deep Learning based Efficient Accident Prevention Technique in Fogged Environment

Deep-APT: Deep Learning based Efficient Accident Prevention Technique in Fogged Environment

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network

Integrating Knowledge Graphs into Autonomous Vehicle Technologies: A Survey of Current State and Future Directions

Contact Info

Product

Resources

About