Transfer Learning for Radio Frequency Machine Learning: A Taxonomy and Survey

Abstract

Transfer learning is a pervasive technology in computer vision and natural language processing fields, yielding exponential performance improvements by leveraging prior knowledge gained from data with different distributions. However, while recent works seek to mature machine learning and deep learning techniques in applications related to wireless communications, a field loosely termed radio frequency machine learning, few have demonstrated the use of transfer learning techniques for yielding performance gains, improved generalization, or to address concerns of training data costs. With modifications to existing transfer learning taxonomies constructed to support transfer learning in other modalities, this paper presents a tailored taxonomy for radio frequency applications, yielding a consistent framework that can be used to compare and contrast existing and future works. This work offers such a taxonomy, discusses the small body of existing works in transfer learning for radio frequency machine learning, and outlines directions where future research is needed to mature the field.

Quantifying and Extrapolating Data Needs in Radio Frequency Machine Learning

“…Depending on assumptions between the source and target, the TL application can be categorized as homogeneous, where differences exist in the distributions between source and target, or heterogeneous, where the differences are in the feature space of the problem [13], [14]. A valuable discussion for understanding the concepts of homogeneous and heterogeneous is provided by Wong and Michaels [15] by explaining the change in distributions as a change of the dataset's collected/generated domain, while the feature space of the problem can be associated with the intended task the source model is trained on and can contrasted to the task of the target problem. The two most common types of TL include retraining the classification head where early layers are frozen during training preserving feature extraction, or fine-tuning of the whole model [16].…”

“…The two most common types of TL include retraining the classification head where early layers are frozen during training preserving feature extraction, or fine-tuning of the whole model [16]. In this work, TL is applied in a homogeneous problem space where the underlying distributions on the data vary, but the generalized problem space is the same between datasets, otherwise called a Domain Adaptation from [15] and more specifically an Environment Platform Co-Adaptation. Additionally, wherever the retraining is done in this work, the fine-tuning approach is utilized, allowing for adjustments to the feature space, which might not be observable in the source dataset.…”

“…The study of TL is complex and well explored [13], [14], [15], [16], [17], [18], and from the effort to understand how to choose an optimal pre-trained model for a desired application, the metrics Negative Conditional Entropy (NCE) [17], Log Expected Empirical Prediction (LEEP) [16], and Logarithm of Maximum Evidence (LogME) [18] are repurposed to analyze the relationship between available data quantity during training and system performance for a given evaluation set.…”

“…To overcome this problem, label whitening to acquire near perfect performance is proposed to act as a quasar that can help map the performance of the metrics with accuracy. The procedure starts with label smoothing [] (14) of the truth labels for the evaluation set, followed by a logit transform (15), which without the label smoothing would not be a useful approach as infinite values would be returned for the correct class and negative infinity for all other classes.…”

Quantifying Raw RF Dataset Similarity for Transfer Learning Applications

“…Such algorithms that utilize raw radio frequency (RF) data as input to ML/DL techniques are considered radio frequency machine learning (RFML) algorithms [1], [2]. Like all traditional ML techniques, most state-of-the-art RFML algorithms require copious amounts of labelled training data drawn from the intended deployment environment, and for the intended deployment environment to remain stable, in order to achieve said state-of-the-art performance [3]. Therefore, recent works have identified transfer learning (TL) as a key research thrust for RFML which would enable developers to train highperforming RFML models quickly and with less labelled data, compared to standard training practices, by using prior knowledge learned on a source domain/task for a target domain/task [3], [4].…”