Micro-Doppler Based Classification of Human Aquatic Activities via Transfer Learning of Convolutional Neural Networks

Abstract

Accurate classification of human aquatic activities using radar has a variety of potential applications such as rescue operations and border patrols. Nevertheless, the classification of activities on water using radar has not been extensively studied, unlike the case on dry ground, due to its unique challenge. Namely, not only is the radar cross section of a human on water small, but the micro-Doppler signatures are much noisier due to water drops and waves. In this paper, we first investigate whether discriminative signatures could be obtained for activities on water through a simulation study. Then, we show how we can effectively achieve high classification accuracy by applying deep convolutional neural networks (DCNN) directly to the spectrogram of real measurement data. From the five-fold cross-validation on our dataset, which consists of five aquatic activities, we report that the conventional feature-based scheme only achieves an accuracy of 45.1%. In contrast, the DCNN trained using only the collected data attains 66.7%, and the transfer learned DCNN, which takes a DCNN pre-trained on a RGB image dataset and fine-tunes the parameters using the collected data, achieves a much higher 80.3%, which is a significant performance boost.

Personnel Recognition and Gait Classification Based on Multistatic Micro-Doppler Signatures Using Deep Convolutional Neural Networks

“…It can be done in two steps: (1) replace the last few layers in the pre-trained network by new designed ones and initialize them randomly; (2) train on the small dataset (referred to as "fine-tuning"). Recent work has explored the feasibility of using CNN pre-trained on optical image dataset to fine-tune micro-Doppler spectrograms [14]. However, the timefrequency spectrograms have only one channel, whereas optical images typically have RGB channels.…”

“…However, the timefrequency spectrograms have only one channel, whereas optical images typically have RGB channels. The authors of [14] simply copy STFT spectrograms for the three input channels to solve this dimension mismatch problem, which is equivalent to regarding the spectrograms as grayscale images. In our method, the STFT spectrograms with three different window sizes are used as different channels of input data.…”

“…In [12], a similar method was used for hand gesture classification. More sophisticated DCNN architectures were used later, including 7-layer DCNN [13], transfer learned AlexNet and VGG-16 network [14] and a three-layer semisupervised auto-encoder [15]. New problems such as low latency classification [16] and multi-target human gait classification [9,17] have also been taken into consideration.…”

Towards Adversarial Denoising of Radar Micro-Doppler Signatures

“…Conversely, Gammacorrection may result in the adverse elimination of low power vital components within the spectrum. This may affect further applications which require knowledge of the local components as well as the overall spectrum structure, such as activity recognition and human subject identification [18]- [20]. Moreover, all of the mentioned denoising techniques are specific to a fixed measurement scenario or SNR level.…”

Human Motion Training Data Generation for Radar Based Deep Learning Applications

“…The raw radar data could be utilized in the time domain without any further processing [25]. The most commonly adopted data type is the microDoppler spectrogram showing the velocity or the Doppler frequency versus time [3], [5], [6], [26]. Moreover, the range versus velocity diagrams at consecutive time steps are used in combination with RNNs [2].…”