Multi-modality Cascaded Fusion Technology for Autonomous Driving

Kuang, Hongwu; Liu, Xiaodong; Zhang, Jingwei; Fang, Zicheng

doi:10.1109/icras49812.2020.9135065

Cited by 9 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, VoxelFusion is better than PointFusion in memory usage. Kuang et al [158] put forth a multi-modality cascaded fusion model for autonomous driving. The fusion model is composed of two parts: intra-frame and inter-frame fusion.…”

Section: Other Fusion Techniquesmentioning

confidence: 99%

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

Alaba,

Gurbuz,

Ball

2024

WEVJ

View full text Add to dashboard Cite

The pursuit of autonomous driving relies on developing perception systems capable of making accurate, robust, and rapid decisions to interpret the driving environment effectively. Object detection is crucial for understanding the environment at these systems’ core. While 2D object detection and classification have advanced significantly with the advent of deep learning (DL) in computer vision (CV) applications, they fall short in providing essential depth information, a key element in comprehending driving environments. Consequently, 3D object detection becomes a cornerstone for autonomous driving and robotics, offering precise estimations of object locations and enhancing environmental comprehension. The CV community’s growing interest in 3D object detection is fueled by the evolution of DL models, including Convolutional Neural Networks (CNNs) and Transformer networks. Despite these advancements, challenges such as varying object scales, limited 3D sensor data, and occlusions persist in 3D object detection. To address these challenges, researchers are exploring multimodal techniques that combine information from multiple sensors, such as cameras, radar, and LiDAR, to enhance the performance of perception systems. This survey provides an exhaustive review of multimodal fusion-based 3D object detection methods, focusing on CNN and Transformer-based models. It underscores the necessity of equipping fully autonomous vehicles with diverse sensors to ensure robust and reliable operation. The survey explores the advantages and drawbacks of cameras, LiDAR, and radar sensors. Additionally, it summarizes autonomy datasets and examines the latest advancements in multimodal fusion-based methods. The survey concludes by highlighting the ongoing challenges, open issues, and potential directions for future research.

show abstract

Section: Other Fusion Techniquesmentioning

confidence: 99%

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

Alaba,

Gurbuz,

Ball

2024

WEVJ

View full text Add to dashboard Cite

show abstract

“…It bolsters functionalities like adaptive cruise control, collision prevention, and lane maintenance assistance, thereby elevating driving safety and efficiency. The amalgamated data facilitates enhanced predictive accuracy and adaptability to evolving road scenarios [18]. However, the implementation of sensor fusion is not without its challenges.…”

Section: Multimodal Fusionmentioning

confidence: 99%

Vehicle Detection in Adverse Weather: A Multi-Head Attention Approach with Multimodal Fusion

Tabassum,

El-Sharkawy

2024

JLPEA

View full text Add to dashboard Cite

In the realm of autonomous vehicle technology, the multimodal vehicle detection network (MVDNet) represents a significant leap forward, particularly in the challenging context of weather conditions. This paper focuses on the enhancement of MVDNet through the integration of a multi-head attention layer, aimed at refining its performance. The integrated multi-head attention layer in the MVDNet model is a pivotal modification, advancing the network’s ability to process and fuse multimodal sensor information more efficiently. The paper validates the improved performance of MVDNet with multi-head attention through comprehensive testing, which includes a training dataset derived from the Oxford Radar RobotCar. The results clearly demonstrate that the multi-head MVDNet outperforms the other related conventional models, particularly in the average precision (AP) of estimation, under challenging environmental conditions. The proposed multi-head MVDNet not only contributes significantly to the field of autonomous vehicle detection but also underscores the potential of sophisticated sensor fusion techniques in overcoming environmental limitations.

show abstract

“…Recent works also look at fusion between Radar and cameras within the perception system. Different Radar representations are proposed to facilitate fusion: spectrogram images [22], sparse locations in image space [29], pseudo-image by projecting to image space [30,4], BEV representation [28] and object detections [16]. However, these methods do not have high accuracy in terms of 3D perception.…”

Section: Related Workmentioning

confidence: 99%

RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects

Yang

Guo

Liang

et al. 2020

Preprint

View full text Add to dashboard Cite

We tackle the problem of exploiting Radar for perception in the context of self-driving as Radar provides complementary information to other sensors such as LiDAR or cameras in the form of Doppler velocity. The main challenges of using Radar are the noise and measurement ambiguities which have been a struggle for existing simple input or output fusion methods. To better address this, we propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxelbased early fusion and an attention-based late fusion, which learn from data to exploit both geometric and dynamic information of Radar data. RadarNet achieves state-of-the-art results on two large-scale real-world datasets in the tasks of object detection and velocity estimation. We further show that exploiting Radar improves the perception capabilities of detecting faraway objects and understanding the motion of dynamic objects.

show abstract

Multi-modality Cascaded Fusion Technology for Autonomous Driving

Cited by 9 publications

References 27 publications

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

Vehicle Detection in Adverse Weather: A Multi-Head Attention Approach with Multimodal Fusion

RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects

Contact Info

Product

Resources

About