Deep Learning Based Image Enhancement for Automotive Radar Trained With an Advanced Virtual Sensor

Abstract

This paper introduces a novel deep learning based concept for image enhancement and distortion suppression in automotive radar signal processing. The deep neural network (DNN) is trained solely on virtual data that is generated by an automotive MIMO radar ray tracing simulator. The simulator mimics the raw data that would be provided by a specific automotive MIMO radar for which the signal processing is envisaged. This virtual radar sensor, which has the same properties as the real radar, creates the DNN training input data. The unique feature of our approach is that an advanced virtual radar sensor is used to create the target values for the DNN training. The advanced virtual sensor is a simulated radar that works on the same basic principle as the real radar and its resembling virtual sensor, but is significantly upgraded in terms of performance. In our experiments the advanced virtual sensor, e.g., has a notably larger array, i.e. a much better lateral resolution than the real radar. In addition, our advanced virtual sensor is not affected by multi-path effects, clutter, noise, beamforming sidelobes and other typical automotive radar distortions. The paper shows that a DNN trained on this advanced virtual sensor training data can deliver outstanding automotive radar signal processing results not only on simulated data but also on field data. The presented real-world automotive radar measurements show that multi-path, clutter and noise are efficiently suppressed by this DNN and that it has a remarkable ability to sharpen images and suppress sidelobes. The results presented in the paper suggest that the novel advanced DNN training concept based on virtual sensors offers opportunities that also go far beyond the radar signal processing capabilities illustrated. Hence, the concept put forward here is an attractive option for placing DNN-based radar signal processing on a novel and easy-to-implement foundation.

Super-Resolution Radar Imaging With Sparse Arrays Using a Deep Neural Network Trained With Enhanced Virtual Data

“…For ULA configurations, this matrix multiplication is equal to a discrete Fourier transform [45] and can therefore be replaced by an FFT. This feature was selected, because it already worked very well in our previous work for radar image enhancement [19]. Instead of only using the magnitude, additional phase information of the signal was selected as feature in this work, as it showed good results in [36].…”

“…The generation of the ground truth signal is described in detail in our earlier work [19]. It is based on a matched filter approach and resembles the description in [46].…”

“…In our previous work [19], we applied a deep neural network (DNN) to radar range-angle images to drastically alleviate clutter and noise in real-world measurement scenarios. This was achieved by simulating both enhanced highresolution noise and clutter-free data as the ground truth and radar data of an accurate digital twin of the radar sensor under test as input.…”

“…Compared to other existing work, our previous work in [19] firstly utilized realistic radar simulations for radar image enhancement with respect of noise and clutter reduction. There and in this work, the simulations were conducted in a realistic environment, resembling a city landscape, see Fig.…”

Synthetic Sensor Measurement Generation With Noise Learning and Multi-Modal Information

“…Furthermore, industries such as manufacturing and industrial automation benefit from synthetic sensor data for optimizing production processes. Simulated sensor data helps in assessing the performance of industrial robots, quality control systems, and predictive maintenance solutions, leading to increased efficiency and cost savings [20], [21]. Moreover, synthetic sensor data plays a role in healthcare applications, particularly in the development of wearable devices and medical sensors.…”