Optimal Transport for Multi-source Domain Adaptation under Target Shift

Redko, Ievgen; Courty, Nicolas; Flamary, Rémi; Tuia, Devis

doi:10.48550/arxiv.1803.04899

Cited by 8 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concerning algorithms, the covariate shift adaptation has been well-studied in the literature [19,17,35]. Importance sampling to address label shift has also been investigated [34], notably with kernel mean matching [39] and Optimal Transport [31]. Recently, a scheme for estimating labels distribution ratio with consistency guarantee has been proposed [21].…”

Section: Related Workmentioning

confidence: 99%

Robust Domain Adaptation: Representations, Weights and Inductive Bias

Bouvier¹,

Véry²,

Chastagnol³

et al. 2020

Preprint

View full text Add to dashboard Cite

Unsupervised Domain Adaptation (UDA) has attracted a lot of attention in the last ten years. The emergence of Domain Invariant Representations (IR) has improved drastically the transferability of representations from a labelled source domain to a new and unlabelled target domain. However, a potential pitfall of this approach, namely the presence of label shift, has been brought to light. Some works address this issue with a relaxed version of domain invariance obtained by weighting samples, a strategy often referred to as Importance Sampling. From our point of view, the theoretical aspects of how Importance Sampling and Invariant Representations interact in UDA have not been studied in depth. In the present work, we present a bound of the target risk which incorporates both weights and invariant representations. Our theoretical analysis highlights the role of inductive bias in aligning distributions across domains. We illustrate it on standard benchmarks by proposing a new learning procedure for UDA. We observed empirically that weak inductive bias makes adaptation more robust. The elaboration of stronger inductive bias is a promising direction for new UDA algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

Robust Domain Adaptation: Representations, Weights and Inductive Bias

Bouvier¹,

Véry²,

Chastagnol³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We will consider two types of shifts between data domains, defining them now as they pertain to image classification. 12,15 First, target shift refers to unequal label prior probabilities between two distributions, p(y) = p (y). The other type is covariate shift, which refers to a difference between the conditional distributions of an image given its class category: p(x|y) = p (x|y).…”

Section: Unsupervised Domain Adaptationmentioning

confidence: 99%

Background adaptive faster R-CNN for semi-supervised convolutional object detection of threats in x-ray images

Sigman

Spell

Liang

et al. 2020

Anomaly Detection and Imaging With X-Rays (ADIX) V

View full text Add to dashboard Cite

Recently, progress has been made in the supervised training of Convolutional Object Detectors (e.g. Faster R-CNN) for threat recognition in carry-on luggage using X-ray images. This is part of the Transportation Security Administration's (TSA's) mission to ensure safety for air travelers in the United States. Collecting more data reliably improves performance for this class of deep algorithm, but requires time and money to produce training data with threats staged in realistic contexts. In contrast to these hand-collected data containing threats, data from the real-world, known as the Stream-of-Commerce (SOC), can be collected quickly with minimal cost; while technically unlabeled, in this work we make a practical assumption that these are without threat objects. Because of these data constraints, we will use both labeled and unlabeled sources of data for the automatic threat recognition problem. In this paper, we present a semi-supervised approach for this problem which we call Background Adaptive Faster R-CNN. This approach is a training method for two-stage object detectors which uses Domain Adaptation methods from the field of deep learning. The data sources described earlier are considered two "domains": one a hand-collected data domain of images with threats, and the other a real-world domain of images assumed without threats. Two domain discriminators, one for discriminating object proposals and one for image features, are adversarially trained to prevent encoding domain-specific information. Penalizing this encoding is important because otherwise the Convolutional Neural Network (CNN) can learn to distinguish images from the two sources based on superficial characteristics, and minimize a purely supervised loss function without improving its ability to recognize objects. For the hand-collected data, only object proposals and image features completely outside of areas corresponding to ground truth object bounding boxes (background) are used. The losses for these domain-adaptive discriminators are added to the Faster R-CNN losses of images from both domains. This technique enables threat recognition based on examples from the labeled data, and can reduce false alarm rates by matching the statistics of extracted features on the hand-collected backgrounds to that of the real world data. Performance improvements are demonstrated on two independently-collected datasets of labeled threats.

show abstract

“…Source Only DANN Ours 5 Related Work Domain Adaptation. The covariate shift adaptation has been studied by [28,25,56,55] and label shift with kernel mean matching [70,20] and Optimal Transport [49]. Since Importance Sampling based methods are limited to distributions which share enough statistical support [29,18], an important line of works focuses on learning domain Invariant Representations (IR) [21,35] for reconciling two non-overlapping data distributions.…”

Section: Source Targetmentioning

confidence: 99%

Target Consistency for Domain Adaptation: when Robustness meets Transferability

Ouali¹,

Bouvier²,

Tami³

et al. 2020

Preprint

View full text Add to dashboard Cite

Learning Invariant Representations has been successfully applied for reconciling a source and a target domain for Unsupervised Domain Adaptation. By investigating the robustness of such methods under the prism of the cluster assumption, we bring new evidence that invariance with a low source risk does not guarantee a wellperforming target classifier. More precisely, we show that the cluster assumption is violated in the target domain despite being maintained in the source domain, indicating a lack of robustness of the target classifier. To address this problem, we demonstrate the importance of enforcing the cluster assumption in the target domain, named Target Consistency (TC), especially when paired with Class-Level InVariance (CLIV). Our new approach results in a significant improvement, on both image classification and segmentation benchmarks, over state-of-the-art methods based on invariant representations. Importantly, our method is flexible and easy to implement, making it a complementary technique to existing approaches for improving transferability of representations. * Equal contribution Preprint. Under review.

show abstract

Optimal Transport for Multi-source Domain Adaptation under Target Shift

Cited by 8 publications

References 14 publications

Robust Domain Adaptation: Representations, Weights and Inductive Bias

Robust Domain Adaptation: Representations, Weights and Inductive Bias

Background adaptive faster R-CNN for semi-supervised convolutional object detection of threats in x-ray images

Target Consistency for Domain Adaptation: when Robustness meets Transferability

Contact Info

Product

Resources

About