Flexible deep transfer learning by separate feature embeddings and manifold alignment

Abstract

Object recognition is a key enabler across industry and defense. As technology changes, algorithms must keep pace with new requirements and data. New modalities and higher resolution sensors should allow for increased algorithm robustness. Unfortunately, algorithms trained on existing labeled datasets do not directly generalize to new data because the data distributions do not match. Transfer learning (TL) or domain adaptation (DA) methods have established the groundwork for transferring knowledge from existing labeled source data to new unlabeled target datasets. However, current DA approaches assume similar source and target feature spaces and suffer in the case of massive domain shifts or changes in the feature space. Existing methods assume the data are either the same modality, or can be aligned to a common feature space. Therefore, most methods are not designed to support a fundamental domain change such as visual to auditory data. We propose a novel deep learning framework that overcomes this limitation by learning separate feature extractions for each domain while minimizing the distance between the domains in a latent lower-dimensional space. The alignment is achieved by considering the data manifold along with an adversarial training procedure. We demonstrate the effectiveness of the approach versus traditional methods with several ablation experiments on synthetic, measured, and satellite image datasets. We also provide practical guidelines for training the network while overcoming vanishing gradients which inhibit learning in some adversarial training settings.

Domain Adaptation by Topology Regularization

“…Recent works by SiSi, 22 Mendoza-Schrock, 23 and Rivera et al 24 have employed optimal transport models to align source and target features in a latent space. In general, optimal transport converts one probability distribution into another.…”

“…Our experiments employ the direct sum domain adversarial transfer (DiSDAT) architecture. 24 This design allows for greater flexibility in the choice of data sets by using separate embeddings for the source and target, then aligning the representations in a common feature space. This network aligns the source and target using Bregman Divergence (BD) Minimization.…”

“…It features a latent space occupied by both the source and target feature representations where various regularization techniques can be applied. We set up our network, using the general CNN architecture of ADDA, 19 but with a latent space dimension of 500, one path for the source and target embedding, and the additional autoencoder of Rivera et al 24 with a parameter of 1. We trained for five Monte-Carlo iterations with 10 epochs each, and did not pre-train on the source only data.…”

“…We begin with a brief discussion of the design of the DiSDAT network. 24 The network learns using a sequence of parameter updates. With each batch, the network first computes the autoencoder loss and accuracy loss.…”

“…Next, the probability distributions of the source and target are aligned using Bregman divergence minimization. We use a divergence penalty following Rivera et al 24 Last, the network trains based on topological loss, attempting to minimize the Wasserstein distance between the source and target persistence diagrams. See Section 4.2 for more information on this loss function and how it was approximated.…”