Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Zhou, Yi; Liu, Lulu; Zhao, Haocheng; López-Benítez, Miguel; Yu, Limin; Yue, Yutao

doi:10.3390/s22114208

Cited by 82 publications

(29 citation statements)

References 216 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A common theme that runs through the conventional approaches to sensing for autonomous operation in complex natural environments has been to counter the lack of controllability and predictability with the collection of large amounts of data. Examples of this approach are the use of vision [49], including monocular [49], stereo [50], and depth cameras (red green blue-depth (RGB-D) cameras, [72]), as well as radar [71] and laser scanning (LIDAR, [38]). The common goal behind applying any of these sensing methods is usually to capture sensory data that would-at least in principle-suffice for building a digital model of the environment that is detailed enough to support planing of autonomous actions.…”

Section: Conventional Approachesmentioning

confidence: 99%

Bioinspiration from bats and new paradigms for autonomy in natural environments

Müller

2024

Bioinspir. Biomim.

View full text Add to dashboard Cite

Achieving autonomous operation in complex natural environment remains an unsolved challenge. Conventional engineering approaches to this problem have focused on collecting large amounts of sensory data that are used to create detailed digital models of the environment. However, this only postpones solving the challenge of identifying the relevant sensory information and linking it to action control to the domain of the digital world model. Furthermore, it imposes high demands in terms of computing power and introduces large processing latencies that hamper autonomous real-time performance. Certain species of bats that are able to navigate and hunt their prey in dense vegetation could be a biological model system for an alternative approach to addressing the fundamental issues associated with autonomy in complex natural environments. Bats navigating in dense vegetation rely on clutter echoes, i.e., signals that consist of unresolved contributions from many scatters. Yet, the animals are able to extract the relevant information from these input signals with brains that are often less than one gram in mass. Pilot results indicate that information relevant to location identification and passageway finding can be directly obtained from clutter echoes, opening up the possibility that the bats' skill can be replicated in man-made autonomous systems.

show abstract

Section: Conventional Approachesmentioning

confidence: 99%

Bioinspiration from bats and new paradigms for autonomy in natural environments

Müller

2024

Bioinspir. Biomim.

View full text Add to dashboard Cite

show abstract

“…In autonomous driving, LiDARs are used to acquire dense point clouds, and each point is generally represented by four values: 3-D coordinates in space and signal intensity. RAD tensors and sparse point clouds are two types of data representations for mm-wave radars [45]. RAD tensors are generated by performing three times fast Fourier transforms (FFTs) on ADC signals in the range, angle, and Doppler velocity dimensions, respectively.…”

Section: A Physical Radarsmentioning

confidence: 99%

RadarVerses in Metaverses: A CPSI-Based Architecture for 6S Radar Systems in CPSS

Liu

Shen

Tian

et al. 2023

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

Metaverses have caused significant changes in the industry and their academic foundation can be traced back to the term cyber-physical-social systems (CPSS), which was proposed in 2010. Radar is an important sensor in sensing systems that are widely applied in many fields, especially, in autonomous driving. To deal with the complex environment, smart radars with realtime information processing capabilities are required. Human factors play a critical role in the operation and management of radar systems, thus, digital twins' radars in cyber-physical systems (CPS) are unable to achieve intelligence in CPSS due to an incomplete consideration of human involvement. For this consideration, we propose a novel framework of RadarVerses for smart radars in metaverses based on ACP-based parallel intelligence, which is also known as cyber-physical-social intelligence (CPSI). RadarVerses consist of five main parts which are physical radars, descriptive radars, predictive radars, prescriptive radars, and deep radars. To construct RadarVerses at the technical level, we introduce four main technical foundations: 1) communication technology; 2) scenarios engineering; 3) foundation models; and 4) digital workers. In addition, we also provide a case study about LiDARs' predictive maintenance of accumulated snow in RadarVerses.

show abstract

“…Because of the unique nature of radar signals and the relative lack of publicly accessible datasets [63] that include both camera and radar datasets [21,[64][65][66][67][68] under foggy weather conditions, the scope of AV research with respect to foggy weather conditions has been severely constrained. For autonomous vehicle research, there are only a very small number of datasets available [67] and [21]which combine information from cameras and radar when undertaken in foggy weather conditions.…”

Section: Datasets and Semantic Labelsmentioning

confidence: 99%

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network

Ogunrinde¹,

Bernadin²

2023

Preprint

View full text Add to dashboard Cite

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster-tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase, and a simple neural network for deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to im-prove tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and cost. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, the speed increased by 37.56%, and identity switches decreased by 46.81%.

show abstract

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Cited by 82 publications

References 216 publications

Bioinspiration from bats and new paradigms for autonomy in natural environments

Bioinspiration from bats and new paradigms for autonomy in natural environments

RadarVerses in Metaverses: A CPSI-Based Architecture for 6S Radar Systems in CPSS

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network

Contact Info

Product

Resources

About