Robust Target Recognition and Tracking of Self-Driving Cars With Radar and Camera Information Fusion Under Severe Weather Conditions

Liu, Ze; Cai, Yingfeng; Wang, Hai; Chen, Long; Gao, Hongbo; Jia, Yunyi; Li, Yicheng

doi:10.1109/tits.2021.3059674

Cited by 184 publications

(76 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It provides a theoretical reference for further practical application. In the follow-up work, some researchers are concerned about how to improve the detection accuracy of [ 7 , 41 , 42 ] at night and under bad weather conditions, and further improvement of the detection accuracy will also be our next research direction.…”

Section: Discussionmentioning

confidence: 99%

“…Generally, autonomous vehicles use various sensors, such as cameras, lidar, and radar, to detect objects [ 5 ]. Some researchers [ 6 ] detect vehicles by extracting binary images from discrete sensor arrays, and some researchers [ 7 ] have achieved good results in the detection task in bad weather through the sensing method of radar and camera information fusion. Compared with other sensors, the camera is now more accurate and more cost-effective at detecting objects.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Real‐Time Object Detector for Autonomous Vehicles Based on YOLOv4

Wang

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Object detection is an important part of autonomous driving technology. To ensure the safe running of vehicles at high speed, real-time and accurate detection of all the objects on the road is required. How to balance the speed and accuracy of detection is a hot research topic in recent years. This paper puts forward a one-stage object detection algorithm based on YOLOv4, which improves the detection accuracy and supports real-time operation. The backbone of the algorithm doubles the stacking times of the last residual block of CSPDarkNet53. The neck of the algorithm replaces the SPP with the RFB structure, improves the PAN structure of the feature fusion module, adds the attention mechanism CBAM and CA structure to the backbone and neck structure, and finally reduces the overall width of the network to the original 3/4, so as to reduce the model parameters and improve the inference speed. Compared with YOLOv4, the algorithm in this paper improves the average accuracy on KITTI dataset by 2.06% and BDD dataset by 2.95%. When the detection accuracy is almost unchanged, the inference speed of this algorithm is increased by 9.14%, and it can detect in real time at a speed of more than 58.47 FPS.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Real‐Time Object Detector for Autonomous Vehicles Based on YOLOv4

Wang

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Many research works reported challenging results when multi-sensor fusion is applied. Integration of LIDAR and camera sensor is presented in [3], [4], [5], [6], and camera and radar fusion is applied in [7], [8].…”

Section: Introductionmentioning

confidence: 99%

LiCaNet: Further Enhancement of Joint Perception and Motion Prediction based on Multi-Modal Fusion

Khalil¹,

Mouftah²

2021

Preprint

View full text Add to dashboard Cite

<p>The safety and reliability of autonomous driving pivots on the accuracy of perception and motion prediction pipelines, which in turn reckons primarily on the sensors deployed onboard. Slight confusion in perception and motion prediction can result in catastrophic consequences due to misinterpretation in later pipelines. Therefore, researchers have recently devoted considerable effort towards developing accurate perception and motion prediction models. To that end, we propose LIDAR Camera network (LiCaNet) that leverages multi-modal fusion to further enhance the joint perception and motion prediction performance accomplished in our earlier work. LiCaNet expands on our previous fusion network by adding a camera image to the fusion of RV image with historical BEV data sourced from a LIDAR sensor. We present a comprehensive evaluation to validate the outstanding performance of LiCaNet compared to the state-of-the-art. Experiments reveal that utilizing a camera sensor results in a substantial perception gain over our previous fusion network and a steep reduction in displacement errors. Moreover, the majority of the achieved improvement falls within camera range, with the highest registered for small and distant objects, confirming the significance of incorporating a camera sensor into a fusion network.</p>

show abstract

“…Conventionally, radar systems were limited to official or governmental entities, but now their smaller form factor, lower cost, higher precision, and easier handling have led to more general utilization. Conventional applications of radars have been aerial [1] and terrestrial [2] traffic control, missile and aerial defense [3], altimetry [4], naval surveillance [5], weather surveillance [6], and astronomy [7], whereas the contemporary radar systems have also been employed in modern medicine [8], autonomous vehicles [9][10][11], geology [12], building security, human activity recognition systems [13][14][15][16], and even in consumer electronics such as mobile phones [17] (serving as a gesture recognition system). It is now safe to assert that the idea of radar sensors being ubiquitous is not far-fetched anymore due to their miniaturization, affordability, and utility.…”

Section: Introductionmentioning

confidence: 99%

Interchannel Interference and Mitigation in Distributed MIMO RF Sensing

Waqar

Pätzold

2021

Sensors

View full text Add to dashboard Cite

In this paper, we analyze and mitigate the cross-channel interference, which is found in multiple-input multiple-output (MIMO) radio frequency (RF) sensing systems. For a millimeter wave (mm-Wave) MIMO system, we present a geometrical three-dimensional (3D) channel model to simulate the time-variant (TV) trajectories of a moving scatterer. We collected RF data using a state-of-the-art radar known as Ancortek SDR-KIT 2400T2R4, which is a frequency-modulated continuous wave (FMCW) MIMO radar system operating in the K-band. The Ancortek radar is currently the only K-band MIMO commercial radar system that offers customized antenna configurations. It is shown that this radar system encounters the problem of interference between the various subchannels. We propose an optimal approach to mitigate the problem of cross-channel interference by inducing a propagation delay in one of the channels and apply range gating. The measurement results prove the effectiveness of the proposed approach by demonstrating a complete elimination of the interference problem. The application of the proposed solution on Ancortek’s SDR-KIT 2400T2R4 allows resolving all subchannel links in a distributed MIMO configuration. This allows using MIMO RF sensing techniques to track a moving scatterer (target) regardless of its direction of motion.

show abstract

Robust Target Recognition and Tracking of Self-Driving Cars With Radar and Camera Information Fusion Under Severe Weather Conditions

Cited by 184 publications

References 26 publications

A Real‐Time Object Detector for Autonomous Vehicles Based on YOLOv4

A Real‐Time Object Detector for Autonomous Vehicles Based on YOLOv4

LiCaNet: Further Enhancement of Joint Perception and Motion Prediction based on Multi-Modal Fusion

Interchannel Interference and Mitigation in Distributed MIMO RF Sensing

Contact Info

Product

Resources

About