A Brief Review of Domain Adaptation

Farahani, Abolfazl; Voghoei, Sahar; Rasheed, Khaled; Arabnia, Hamid R.

doi:10.1007/978-3-030-71704-9_65

Cited by 312 publications

(131 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examples are BERTje and RobBERT [19] [3], the Dutch equivalents of BERT and RoBERTa, respectively), and CamemBERT [14], a French BERT model. Domain adaptation, a special case of transfer learning where the model is first trained on unsupervised data from the domain of an intended task, aims to improve results even further "by minimizing the difference between the domain distributions" ( [6], p. 1), thus creating a model that optimally learns from the training data. Regarding the domain of COVID-19, COVID-Twitter-BERT [16], a BERT-large model pre-trained on COVID-19 tweets, has shown statistically significant gains over the baseline BERT-large in various applications, including vaccine stance classification.…”

Section: Related Researchmentioning

confidence: 99%

CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection

Lemmens¹,

Nooten²,

Kreutz³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present CoNTACT 1 : a Dutch language model adapted to the domain of COVID-19 tweets. The model was developed by continuing the pre-training phase of RobBERT [3] by using 2.8M Dutch COVID-19 related tweets posted in 2021. In order to test the performance of the model and compare it to RobBERT, the two models were tested on two tasks: (1) binary vaccine hesitancy detection and (2) detection of arguments for vaccine hesitancy. For both tasks, not only Twitter but also Facebook data was used to show cross-genre performance. In our experiments, CoNTACT showed statistically significant gains over RobBERT in all experiments for task 1. For task 2, we observed substantial improvements in virtually all classes in all experiments. An error analysis indicated that the domain adaptation yielded better representations of domain-specific terminology, causing CoNTACT to make more accurate classification decisions.

show abstract

Section: Related Researchmentioning

confidence: 99%

CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection

Lemmens¹,

Nooten²,

Kreutz³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As a solution adopted from transfer learning method, Domain Adaptation (DA) [108] tries to reduce disparity between source and target domain, improving model's performance on unseen data. This technique is applicable when the training data does not accurately have the same distribution as test data which is quite often in real-world applications and results in performance degradation.…”

Section: Domain Adaptationmentioning

confidence: 99%

Deep Learning meets Liveness Detection: Recent Advancements and Challenges

Sabaghi¹,

Oghbaie²,

Kooshan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Facial biometrics has been recently received tremendous attention as a convenient replacement for traditional authentication systems. Consequently, detecting malicious attempts has found great significance, leading to extensive studies in face anti-spoofing (FAS),i.e., face presentation attack detection. Deep feature learning and techniques, as opposed to hand-crafted features, have promised dramatic increase in the FAS systems' accuracy, tackling the key challenges of materializing realworld application of such systems. Hence, a new research area dealing with development of more generalized as well as accurate models is increasingly attracting the attention of the research community and industry. In this paper, we present a comprehensive survey on the literature related to deep-feature-based FAS methods since 2017. To shed light on this topic, a semantic taxonomy based on various features and learning methodologies is represented. Further, we cover predominant public datasets for FAS in a chronological order, their evolutional progress, and the evaluation criteria (both intra-dataset and inter-dataset). Finally, we discuss the open research challenges and future directions.

show abstract

“…Previous works perform disentanglement using paraphrase pairs as information for semantics, and/or constituency parses as information for syntax. The 1 github.com/ghazi-f/QKVAE dependence of models on labeled data is known to entail high cost (see Seddah et al, 2020 on syntactic annotation), and to often require new labels to handle problems such as concept drift (Lu et al, 2019) and domain adaptation (Farahani et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs

Felhi¹,

Roux²,

Seddah³

2022

Preprint

View full text Add to dashboard Cite

We propose a generative model for text generation, which exhibits disentangled latent representations of syntax and semantics. Contrary to previous work, this model does not need syntactic information such as constituency parses, or semantic information such as paraphrase pairs. Our model relies solely on the inductive bias found in attention-based architectures such as Transformers.In the attention of Transformers, keys handle information selection while values specify what information is conveyed. Our model, dubbed QKVAE, uses Attention in its decoder to read latent variables where one latent variable infers keys while another infers values.We run experiments on latent representations and experiments on syntax/semantics transfer which show that QKVAE displays clear signs of disentangled syntax and semantics. We also show that our model displays competitive syntax transfer capabilities when compared to supervised models and that comparable supervised models need a fairly large amount of data (more than 50K samples) to outperform it on both syntactic and semantic transfer. The code for our experiments is publicly available 1 .

show abstract

A Brief Review of Domain Adaptation

Cited by 312 publications

References 48 publications

CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection

CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection

Deep Learning meets Liveness Detection: Recent Advancements and Challenges

Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs

Contact Info

Product

Resources

About