Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup

Ray, Chowdhury, Jishnu; Caragea, Cornelia; Caragea, Doina

doi:10.18653/v1/2020.acl-srw.39

Cited by 22 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For MLP, we represent an input example as the average of the embeddings of its tokens, and denote the method as MLP-BoW. We also use DenseCNN [14] and XML-CNN [18] following [23] for disaster and DPCNN [10] for sentiment datasets. The results are presented in Table 4.…”

Section: Resultsmentioning

confidence: 99%

Supervised Graph Contrastive Pretraining for Text Classification

Ghosh,

Maji,

Desarkar

2021

Preprint

View full text Add to dashboard Cite

Contrastive pretraining techniques for text classification has been largely studied in an unsupervised setting. However, oftentimes labeled data from related tasks which share label semantics with current task is available. We hypothesize that using this labeled data effectively can lead to better generalization on current task. In this paper, we propose a novel way to effectively utilize labeled data from related tasks with a graph based supervised contrastive learning approach. We formulate a token-graph by extrapolating the supervised information from examples to tokens. Our formulation results in an embedding space where tokens with high/low probability of belonging to same class are near/further-away from one another. We also develop detailed theoretical insights which serve as a motivation for our method. In our experiments with 13 datasets, we show our method outperforms pretraining schemes by 2.5% and also example-level contrastive learning based formulation by 1.8% on average. In addition, we show cross-domain effectiveness of our method in a zero-shot setting by 3.91% on average. Lastly, we also demonstrate our method can be used as a noisy teacher in a knowledge distillation setting to significantly improve performance of transformer based models in low labeled data regime by 4.57% on average.

show abstract

Section: Resultsmentioning

confidence: 99%

Supervised Graph Contrastive Pretraining for Text Classification

Ghosh,

Maji,

Desarkar

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We highlight a few works which explore disaster-related tweet classification in multilingual setting. One of the comprehensive works in this area was done by Raychowdhury et al in [25] which explore disaster-related text classification by applying Manifold Mixup [28] on mBERT. They aggregated multiple disaster datasets containing tweets in multiple languages into a single large dataset and performed their experiments on that dataset.…”

Section: Related Workmentioning

confidence: 99%

Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification

Ghosh¹,

Maji²,

Desarkar³

2022

Preprint

View full text Add to dashboard Cite

Online social media works as a source of various valuable and actionable information during disasters. These information might be available in multiple languages due to the nature of user generated content. An effective system to automatically identify and categorize these actionable information should be capable to handle multiple languages and under limited supervision. However, existing works mostly focus on English language only with the assumption that sufficient labeled data is available. To overcome these challenges, we propose a multilingual disaster related text classification system which is capable to work undervmonolingual, cross-lingual and multilingual lingual scenarios and under limited supervision. Our end-to-end trainable framework combines the versatility of graph neural networks, by applying over the corpus, with the power of transformer based large language models, over examples, with the help of cross-attention between the two. We evaluate our framework over total nine English, Non-English and monolingual datasets invmonolingual, cross-lingual and multilingual lingual classification scenarios. Our framework outperforms state-of-the-art models in disaster domain and multilingual BERT baseline in terms of Weighted F 1 score. We also show the generalizability of the proposed model under limited supervision.

show abstract

“…With regard to input representations, crisis-specific embeddings have been shown to be particularly effective (Nguyen et al, 2016). Recently, work has shown the effectiveness of deep contextualized word representations for classifying disaster-based tweets (Madichetty and M, 2020), particularly with regard to classifying over multiple disaster types (Zahera et al, 2019;Ray Chowdhury et al, 2020).…”

Section: Classification In Crisismentioning

confidence: 99%

The challenges of temporal alignment on Twitter during crises

Stowe¹,

Gurevych²

2021

Preprint

View full text Add to dashboard Cite

Language usage changes over time, and this can impact the effectiveness of NLP systems. This work investigates methods for adapting to changing discourse during crisis events. We explore social media data during crisis, for which effective, time-sensitive methods are necessary. We experiment with two separate methods to accommodate changing data: temporal pretraining, which uses unlabeled data for the target time periods to train better language models, and a model of embedding shift based on tools for analyzing semantic change. This shift allows us to counteract temporal drift by normalizing incoming data based on observed patterns of language change. Simulating scenarios in which we lack access to incoming labeled data, we demonstrate the effectiveness of these methods for a wide variety of crises, showing we can improve performance by up to 8.0 F1 score for relevance classification across datasets.

show abstract

Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup

Cited by 22 publications

References 23 publications

Supervised Graph Contrastive Pretraining for Text Classification

Supervised Graph Contrastive Pretraining for Text Classification

Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification

The challenges of temporal alignment on Twitter during crises

Contact Info

Product

Resources

About