Domain Adaptation with Adversarial Training and Graph Embeddings

Alam, Firoj; Joty, Shafiq; Imran, Muhammad

doi:10.18653/v1/p18-1099

Cited by 95 publications

(68 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DANNs have been applied in many NLP tasks in the last few years, mainly to sentiment classification (e.g., Ganin et al (2016), Li et al (2018a), Shen et al (2018), Rocha andLopes Cardoso (2019), Ghoshal et al (2020), to name a few), but recently to many other tasks as well: language identification (Li et al, 2018a), natural language inference (Rocha and Lopes Cardoso, 2019), POS tagging (Yasunaga et al, 2018), parsing (Sato et al, 2017), trigger identification (Naik and Rose, 2020), relation extraction Fu et al, 2017;Rios et al, 2018), and other (binary) text classification tasks like relevancy identification (Alam et al, 2018a), machine reading comprehension , stance detection (Xu et al, 2019), and duplicate question detection (Shah et al, 2018). This makes DANNs the most widely used UDA approach in NLP, as illustrated in Table 1.…”

Section: Domain Adversariesmentioning

confidence: 99%

“…Work on the intersection of data-centric and model-centric methods can be plentiful. It currently includes combining semi-supervised objectives with an adversarial loss (Lim et al, 2020;Alam et al, 2018b), combining pivot-based approaches with pseudo-labeling (Cui and Bollegala, 2019) and very recently with contextualized word embeddings (Ben-David et al, 2020), and combining multi-task approaches with domain shift (Jia et al, 2019), multi-task learning with pseudo-labeling (multi-task tritraining) (Ruder and Plank, 2018), and adaptive ensembling (Desai et al, 2019), which uses a studentteacher network with a consistency-based self-ensembling loss and a temporal curriculum. They apply adaptive ensembling to study temporal and topic drift in political data classification (Desai et al, 2019).…”

Section: Hybrid Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Neural Unsupervised Domain Adaptation in NLP—A Survey

Ramponi¹,

Plank²

2020

Proceedings of the 28th International Conference on Computational Linguistics

154

108

View full text Add to dashboard Cite

Deep neural networks excel at learning from labeled data and achieve state-of-the-art results on a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge. Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP. 1

show abstract

Section: Domain Adversariesmentioning

confidence: 99%

Section: Hybrid Approachesmentioning

confidence: 99%

Neural Unsupervised Domain Adaptation in NLP—A Survey

Ramponi¹,

Plank²

2020

Proceedings of the 28th International Conference on Computational Linguistics

154

108

View full text Add to dashboard Cite

show abstract

“…Li et al (2018) and Mazloom et al (2019) showed that models adapted to the domain of the event perform better than generalized models. Alam et al (2018a) propose an interesting variant for neural networks: Their system includes an adversarial component which can be used to adapt a model trained on a specific event to a new one (i.e. a new domain).…”

Section: Machine Learning Approachesmentioning

confidence: 99%

Review article: Detection of informative tweets in crisis events

Kruspe

Kersten

Klan

2020

Preprint

View full text Add to dashboard Cite

Abstract. Messages on social media can be an important source of information during crisis situations, be they short-term disasters or longer-term events like COVID-19. They can frequently provide details about developments much faster than traditional sources (e.g. official news) and can offer personal perspectives on events, such as opinions or specific needs. In the future, these messages can also serve to assess disaster risks. One challenge for utilizing social media in crisis situations is the reliable detection of informative messages in a flood of data. Researchers have started to look into this problem in recent years, beginning with crowd-sourced methods. Lately, approaches have shifted towards an automatic analysis of messages. In this review article, we present methods for the automatic detection of crisis-related messages (tweets) on Twitter. We start by showing the varying definitions of importance and relevance relating to disasters, as they can serve very different purposes. This is followed by an overview of existing, crisis-related social media data sets for evaluation and training purposes. We then compare approaches for solving the detection problem based (1) on filtering by characteristics like keywords and location, (2) on crowdsourcing, and (3) on machine learning techniques with regard to their focus, their data requirements, their technical prerequisites, their efficiency and accuracy, and their time scales. These factors determine the suitability of the approaches for different expectations, but also their limitations. We identify which aspects each of them can contribute to the detection of informative tweets, and which areas can be improved upon in the future.We point out particular challenges, such as the linguistic issues concerning this kind of data. Finally, we suggest future avenues of research, and show connections to related tasks, such as the subsequent semantic classification of tweets.

show abstract

“…Finally, transfer learning is a central agenda in this paper; an excellent survey of dominant techniques may be found in [4]. More recent work on domain adaptation may be found in [28], with the work in [29] applied specifically to the disaster response problem. Pedrood and Purohit [29] also applied transfer learning to the problem of mining help intent on Twitter.…”

Section: Related Workmentioning

confidence: 99%

Low-supervision urgency detection and transfer in short crisis messages

Kejriwal

Zhou

2019

Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining

View full text Add to dashboard Cite

Humanitarian disasters have been on the rise in recent years due to the effects of climate change and sociopolitical situations such as the refugee crisis. Technology can be used to best mobilize resources such as food and water in the event of a natural disaster, by semi-automatically flagging tweets and short messages as indicating an urgent need. The problem is challenging not just because of the sparseness of data in the immediate aftermath of a disaster, but because of the varying characteristics of disasters in developing countries (making it difficult to train just one system) and the noise and quirks in social media. In this paper, we present a robust, low-supervision social media urgency system that adapts to arbitrary crises by leveraging both labeled and unlabeled data in an ensemble setting. The system is also able to adapt to new crises where an unlabeled background corpus may not be available yet by utilizing a simple and effective transfer learning methodology. Experimentally, our transfer learning and lowsupervision approaches are found to outperform viable baselines with high significance on myriad disaster datasets.

show abstract

Domain Adaptation with Adversarial Training and Graph Embeddings

Cited by 95 publications

References 20 publications

Neural Unsupervised Domain Adaptation in NLP—A Survey

Neural Unsupervised Domain Adaptation in NLP—A Survey

Review article: Detection of informative tweets in crisis events

Low-supervision urgency detection and transfer in short crisis messages

Contact Info

Product

Resources

About