Domain-Adversarial Training of Neural Networks

Ganin, Yaroslav; Ustinova, Evgeniya; Ajakan, Hana; Germain, Pascal; Larochelle, Hugo; Laviolette, François; Marchand, Mario; Lempitsky, Victor

doi:10.48550/arxiv.1505.07818

Cited by 72 publications

(84 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been recent advances in this field through the use of generative adversarial networks (GANs, see e.g. [14]), which aims to align the domains through unsupervised or semi-supervised learning while retaining good performance on the source domain classification.…”

Section: Resultsmentioning

confidence: 99%

Improved Photometric Classification of Supernovae using Deep Learning

Moss

2018

Preprint

View full text Add to dashboard Cite

We present improved photometric supernovae classification using deep recurrent neural networks. The main improvements over previous work are (i) the introduction of a time gate in the recurrent cell that uses the observational time as an input; (ii) greatly increased data augmentation including time translation, addition of Gaussian noise and early truncation of the lightcurve. For post Supernovae Photometric Classification Challenge (SPCC) data, using a training fraction of 5.2% (1103 supernovae) of a representational dataset, we obtain a type Ia vs. non type Ia classification accuracy of 93.2 ± 0.1%, a Receiver Operating Characteristic curve AUC of 0.980 ± 0.002 and a SPCC figureof-merit of F1 = 0.57±0.01. Using a representational dataset of 50% (10, 660 supernovae), we obtain a classification accuracy of 96.6 ± 0.1%, an AUC of 0.995 ± 0.001 and F1 = 0.76 ± 0.01. We found the non-representational training set of the SPCC resulted in a large degradation in performance due to a lack of faint supernovae, but this can be alleviated by the introduction of only a small number (∼ 100) of faint training samples. We also outline ways in which this could be achieved using unsupervised or semi-supervised domain adaptation.

show abstract

Section: Resultsmentioning

confidence: 99%

Improved Photometric Classification of Supernovae using Deep Learning

Moss

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Researchers have encouraged invariance or forgetting with VAEs by utilizing various combinations of weak supervision [1], [23], adversarial training [6], and by trading off reconstruction error against encoding capacity according to the information bottleneck principle [25]. Others have sought to fully disentangle the latent space into independent partitions [14], thereby enabling the isolation of wanted from unwanted information.…”

Section: Literature Reviewmentioning

confidence: 99%

Improving Robot Localisation by Ignoring Visual Distraction

Méndez¹,

Vowels²,

Bowden³

2021

Preprint

View full text Add to dashboard Cite

Attention is an important component of modern deep learning. However, less emphasis has been put on its inverse: ignoring distraction. Our daily lives require us to explicitly avoid giving attention to salient visual features that confound the task we are trying to accomplish. This visual prioritisation allows us to concentrate on important tasks while ignoring visual distractors.In this work, we introduce Neural Blindness, which gives an agent the ability to completely ignore objects or classes that are deemed distractors. More explicitly, we aim to render a neural network completely incapable of representing specific chosen classes in its latent space. In a very real sense, this makes the network "blind" to certain classes, allowing and agent to focus on what is important for a given task, and demonstrates how this can be used to improve localisation.

show abstract

“…In contrast, we show that the first layers are not immune to domain shifts even though they are generic feature extractors and that, by improving only the first layer features, we can already improve the performance of the whole network. Finally, [1] shows a significant improvement through the use of a deep network along with a domain regressor. This is done by adding a sub-network consisting of parallel layers to the classification layer.…”

Section: Shallow Damentioning

confidence: 99%

“…Mnist→ MnistM: here, we train a network on Mnist dataset for hand written digits recognition that are black & white, while the target test is the MnistM dataset, composed of the same digits as Mnist [9] but blended with patches from real image [31]. We followed the procedure described by [1] and further verified by having the same results. Synthetic traffic signs→ Dark illumination: in this setting, we want to imitate the real life condition in which we train a system on a large dataset of synthetic images that could as much as possible mimic all the different conditions, and then test it on a dataset affected by a domain shift.…”

Section: Setup and Datasetsmentioning

confidence: 99%

“…It is the task of adapting a system trained on one data set (the source S) to be functional on a different data set (the target T). State-of-the-art methods for unsupervised domain adaptation of deep neural network architectures [1,2] proceed by adding new layers to the deep network or learning a joint architecture in order to come up with representations that are more general and informative across the source and target domains. We will refer to these methods, that retrain the network using the information coming from the new (unlabeled) target samples, as deep adaptation methods.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction

Aljundi¹,

Tuytelaars²

2016

Preprint

View full text Add to dashboard Cite

End-to-end learning methods have achieved impressive results in many areas of computer vision. At the same time, these methods still suffer from a degradation in performance when testing on new datasets that stem from a different distribution. This is known as the domain shift effect. Recently proposed adaptation methods focus on retraining the network parameters. However, this requires access to all (labeled) source data, a large amount of (unlabeled) target data, and plenty of computational resources. In this work, we propose a lightweight alternative, that allows adapting to the target domain based on a limited number of target samples in a matter of minutes rather than hours, days or even weeks. To this end, we first analyze the output of each convolutional layer from a domain adaptation perspective. Surprisingly, we find that already at the very first layer, domain shift effects pop up. We then propose a new domain adaptation method, where first layer convolutional filters that are badly affected by the domain shift are reconstructed based on less affected ones. This improves the performance of the deep network on various benchmark datasets.

show abstract

Domain-Adversarial Training of Neural Networks

Cited by 72 publications

References 0 publications

Improved Photometric Classification of Supernovae using Deep Learning

Improved Photometric Classification of Supernovae using Deep Learning

Improving Robot Localisation by Ignoring Visual Distraction

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction

Contact Info

Product

Resources

About