Unsupervised Multi-source Domain Adaptation for Regression

Abstract

We consider the problem of unsupervised domain adaptation from multiple sources in a regression setting. We propose in this work an original method to take benefit of different sources using a weighted combination of the sources. For this purpose, we define a new measure of similarity between probabilities for domain adaptation which we call hypothesisdiscrepancy. We then prove a new bound for unsupervised domain adaptation combining multiple sources. We derive from this bound a novel adversarial domain adaptation algorithm adjusting weights given to each source, ensuring that sources related to the target receive higher weights. We finally evaluate our method on different public datasets and compare it to other domain adaptation baselines to demonstrate the improvement for regression tasks.

Adversarial Weighting for Domain Adaptation in Regression

“…This process aims to minimize the H-divergence [1]. Since this seminal work, many domain adaptation methods based on adversarial networks have been proposed with other metrics as the Wasserstein's [27], the discrepancy [26], the disparity discrepancy [35] and the hdiscrepancy [23]. The latter work highlights the advantage of using discrepancy to handle regression tasks over the Hdivergence.…”

“…Adversarial domain adaptation methods were originally introduced to learn an encoding space in which the source and target data can not be distinguished by any domain classifier, with the encoder and domain classifier trained with opposing objectives [13]. Since then, adversarial training has been used as a powerful tool to minimize complex losses defined as maxima on functional spaces [23], [26], [27], [35]. The success of adversarial methods lies in their computational speed and ability to scale to large data sets through the use of stochastic gradient descent-ascent algorithms.…”

“…The success of adversarial methods lies in their computational speed and ability to scale to large data sets through the use of stochastic gradient descent-ascent algorithms. Moreover, the efficiency of adversarial methods leads to significant improvements for many tasks as image classification [35], image segmentation [26] or sentiment analysis [23].…”

“…These reasons motivate our choice to use a purely supervised metric between source and target distributions: the Y-discrepancy. Furthermore, discrepancy metrics are suited for regression task as they provide theoretical adaptation guarantees on the target risk for common regression losses (as the L p losses) [9], [18], [19], [23]. We propose an original algorithm called Weighting Adversarial Neural Network (WANN) which relies on an adversarial weighting approach to minimize the Y-discrepancy.…”

Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport

“…The DAT approach has been applied and confirmed to effectively compensate for the mismatch of source (training time) and target (testing time) conditions in numerous tasks, such as speech signal processing [19,20], image processing [15,21], and wearable sensor signal processing [22]. A later development in Multisource Domain Adversarial Networks (MDAN) [23] extended the original DAT to lift the constraint of single domain transition, utilizing multiple domain classifiers to extract discriminative deep features for the main learning task while being invariant to multiple domain shifts [3,24].…”