Transfer Learning from Audio Deep Learning Models for Micro-Doppler Activity Recognition

Tran, Kimberly T.; Griffin, Lewis D.; Chetty, Kevin; Vishwakarma, Shelly

doi:10.1109/radar42522.2020.9114643

Cited by 11 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [8], a ResNet based pretrained CNN on ImageNet database, is fine-tuned on time-Doppler spectrograms. The authors of [30] proposed a generative adversarial based image-to-image translation approach to transform time-Doppler signatures into a pseudo-audio representation, and fine-tuned a pre-trained VGGish CNN to classify the obtained representations.…”

Section: Related Workmentioning

confidence: 99%

Multi-View CNN-LSTM Architecture for Radar-Based Human Activity Recognition

et al. 2022

View full text Add to dashboard Cite

In this paper, we propose a multi-view Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) network which fuses multiple "views" of the time-range-Doppler radar data-cube for human activity recognition. It adopts the structure of convolutional neural networks to extract optimal frame based features from the time-range, time-Doppler and range-Doppler projections of the radar datacube. The CNN models are trained using an unsupervised Convolutional Auto-Encoder (CAE) topology. Afterwards, the pre-trained parameters of the encoder are fine-tuned to extract intermediate frame based representations, which are subsequently aggregated via LSTM networks for sequence classification. The temporal correlation among the views is explicitly learned by sharing the LSTM network weights across different views. Moreover, we propose range and Doppler energy dispersion and temporal difference based features as an input to the CNN-LSTM models. Furthermore, we investigate the use of target tracking features as an auxiliary side information. The proposed model is trained on datasets collected in both cluttered and uncluttered environments. For validation, an independent test dataset, with unseen participants, in a cluttered environment was collected. Fusion with auxiliary features improves the generalization by 5%, yielding an overall Macro F1-score of 74.7%.

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-View CNN-LSTM Architecture for Radar-Based Human Activity Recognition

et al. 2022

View full text Add to dashboard Cite

show abstract

“…While relatively simple CNNs consisting of only three to five convolutional layers have been shown to be successful, deeper and more complex network approaches have also been investigated [26]. Furthermore, more advanced deep learning approaches have also been taken to compensate for limited training data, such as generating data using generative adversarial networks and transfer learning [22,30].…”

Section: Related Workmentioning

confidence: 99%

“…While we observe that the specific model we used outperforms on this data set and generalizes well, this is not necessarily the case for larger networks or different data sets. For example, in [30], the authors find that the SVM of the spectrogram outperforms much deeper networks like VGG-16. Additionally, unlike CNNs, features defined on a specific manifold through the kernel function provide a clear advantage in being able to more precisely obtain invariance to well defined nuisance transformations known a priori, based on knowledge of the application and data.…”

Section: Related Workmentioning

confidence: 99%

A manifold learning approach for gesture recognition from micro-Doppler radar measurements

Mason¹,

Mhaskar²,

Guo³

2021

Preprint

View full text Add to dashboard Cite

A recent paper (Neural Networks, 132 (2020), 253-268) introduces a straightforward and simple kernel based approximation for manifold learning that does not require the knowledge of anything about the manifold, except for its dimension. In this paper, we examine the pointwise error in approximation using least squares optimization based on this kernel, in particular, how the error depends upon the data characteristics and deteriorates as one goes away from the training data. The theory is presented with an abstract localized kernel, which can utilize any prior knowledge about the data being located on an unknown sub-manifold of a known manifold.We demonstrate the performance of our approach using a publicly available micro-Doppler data set investigating the use of different pre-processing measures, kernels, and manifold dimension. Specifically, it is shown that the Gaussian kernel introduced in the above mentioned paper leads to a near-competitive performance to deep neural networks, and offers significant improvements in speed and memory requirements. Similarly, a kernel based on treating the feature space as a submanifold of the Grassman manifold outperforms conventional hand-crafted features. To demonstrate the fact that our methods are agnostic to the domain knowledge, we examine the classification problem in a simple video data set.

show abstract

“…At its simplest, the first n convolutional layers and their weights from the feature extraction part of an existing model are copied to the first n layers of a new model for a related or similar task, with the remaining layers either re‐initialized with randomized weights or replaced (e.g., Razavian et al., 2014; Yosinski et al., 2014). These tasks need not be near‐identical or even superficially related, as long as low‐level data characteristics are shared between tasks (e.g., Efremova et al., 2019; Tran et al., 2020; Zamir et al., 2018). The intuition is that generalized knowledge of data structure and properties from one model trained with abundant labeled data (or “big data”) can guide a learning algorithm toward a good solution for a new task with far more limited, or even no, labeled data.…”

Section: Introductionmentioning

confidence: 99%

A Little Data Goes a Long Way: Automating Seismic Phase Arrival Picking at Nabro Volcano With Transfer Learning

et al. 2021

View full text Add to dashboard Cite

Seismic monitoring plays a fundamental part in mitigating hazards at volcanoes. During periods of unrest, thousands of earthquakes can occur each day, producing a diverse range of seismic signals that reflect a multitude of interlinked volcanic processes (e.g., migrating fluids, fault movement, explosions, rockfalls). These earthquakes are generally recorded by broadband seismometers, which are highly sensitive to ground motion across a wide range of frequencies and record signals at high sample rates (typically 100 times or more per second). This level of detail, however, comes at the cost of generating vast amounts of data. Many seismic networks utilize tens or even hundreds of seismometers at a given time (e.g., Hansen & Schmandt, 2015), making real-time manual inspection of these time series practically infeasible. Previous

show abstract

Transfer Learning from Audio Deep Learning Models for Micro-Doppler Activity Recognition

Cited by 11 publications

References 14 publications

Multi-View CNN-LSTM Architecture for Radar-Based Human Activity Recognition

Multi-View CNN-LSTM Architecture for Radar-Based Human Activity Recognition

A manifold learning approach for gesture recognition from micro-Doppler radar measurements

A Little Data Goes a Long Way: Automating Seismic Phase Arrival Picking at Nabro Volcano With Transfer Learning

Contact Info

Product

Resources

About