Adaptive Ensembling: Unsupervised Domain Adaptation for Political Document Analysis

Abstract

Insightful findings in political science often require researchers to analyze documents of a certain subject or type, yet these documents are usually contained in large corpora that do not distinguish between pertinent and nonpertinent documents. In contrast, we can find corpora that label relevant documents but have limitations (e.g., from a single source or era), preventing their use for political science research. To bridge this gap, we present adaptive ensembling, an unsupervised domain adaptation framework, equipped with a novel text classification model and time-aware training to ensure our methods work well with diachronic corpora. Experiments on an expert-annotated dataset show that our framework outperforms strong benchmarks. Further analysis indicates that our methods are more stable, learn better representations, and extract cleaner corpora for fine-grained analysis.

Neural Unsupervised Domain Adaptation in NLP—A Survey

“…For example, classic methods such as tri-training constitute a strong baseline for domain shift in neural times (Ruder and Plank, 2018). Pseudo-labeling has recently been studied for parsing with contextualized word representations (Rotman and Reichart, 2019;Lim et al, 2020) and a recent work proposes adaptive ensembling (Desai et al, 2019) as extension of temporal ensembling (see hybrid methods in Section 6).…”

“…Work on the intersection of data-centric and model-centric methods can be plentiful. It currently includes combining semi-supervised objectives with an adversarial loss (Lim et al, 2020;Alam et al, 2018b), combining pivot-based approaches with pseudo-labeling (Cui and Bollegala, 2019) and very recently with contextualized word embeddings (Ben-David et al, 2020), and combining multi-task approaches with domain shift (Jia et al, 2019), multi-task learning with pseudo-labeling (multi-task tritraining) (Ruder and Plank, 2018), and adaptive ensembling (Desai et al, 2019), which uses a studentteacher network with a consistency-based self-ensembling loss and a temporal curriculum. They apply adaptive ensembling to study temporal and topic drift in political data classification (Desai et al, 2019).…”

“…Comprehensive UDA benchmarks Concretely, we recommend a) to create new benchmarks for UDA with multiple tasks and of increasing complexity, setups beyond 1:1 adaptation, and datasets which document known variety facets of the data (Bender and Friedman, 2018). This will help to learn about the known and unknown (Section 3) as 'variety' (domain) matters; b) to release unlabeled data from the broader distribution from which annotated data was sampled, in line with Gururangan et al (2020); this allows studying diachronic effects, as labeled evaluation data lacks diversity in terms of topics and time (Desai et al, 2019;Derczynski et al, 2016); and c) to release unaggregated, multiple annotations to study divergences in annotations (Plank et al, 2014a).…”

Political Ideology and Polarization: A Multi-dimensional Approach

“…Raw data Since polarization is a process that needs to be analyzed over time (DiMaggio et al, 1996), our annotated articles are sampled from a diachronic corpus of 1,749 news articles across nearly 3 decades (from 1947 till 1974). Articles in this corpus are from political news articles of Desai et al (2019) from the Corpus of Historical American English (COHA, Davies ( 2012)) covering years 1922-1986. These 1,749 articles are extracted such that: (1) they cover broad and politically relevant topics (ranging from education and health to economy) but still share discussions related to the federal budget to make our annotations tractable 4 ; (2) balanced in the number of articles across 5 news outlets with center-left, central, and center-right ideology (c.f.…”

Unsupervised Domain Adaptation with Global and Local Graph Neural Networks in Limited Labeled Data Scenario: Application to Disaster Management

“…• Feature projection [5], [8], [9] • Instance re-weighting [10]- [12] • Pivot feature centric [13], [14] • Domain Adversarial / Gradient Reversal based [6], [15]- [19] Feature projection signifies bringing the features of source and target domain to a joint latent space. [8] used stacked autoencoders to learn domain adaptive feature representations for sentiment analysis.…”