Domain-Guided Task Decomposition with Self-Training for Detecting Personal Events in Social Media

Karisani, Payam; Ho, Joyce C.; Agichtein, Eugene

doi:10.1145/3366423.3380304

Cited by 13 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A neural network variant of co-training [9] is proposed in [42]. In [30], the authors propose a framework to integrate human knowledge with co-training. In [55], a reinforcement learning variant of co-training is proposed.…”

Section: Related Workmentioning

confidence: 99%

“…Co-Decomp. We included the framework introduced in [30]. This model uses domain knowledge to decompose the task into a set of subtasks to be solved in a multi-view setting.…”

Section: Baselinesmentioning

confidence: 99%

“…In this study we evaluated Self-Pretraining in three Twitter datasets. We selected strong baselines, i.e., Tri-training with disagreement [50], Mutual Learning [61], Spaced Repetition [4], and Co-Decomp [30], and showed that some of them fail under certain cases. As opposed to these models, we demonstrated that Self-Pretraining is either the best model or on a par with the best model in every setting.…”

Section: Empirical Analysismentioning

confidence: 99%

See 2 more Smart Citations

Semi-Supervised Text Classification via Self-Pretraining

Karisani

2021

Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

We present a neural semi-supervised learning model termed Self-Pretraining. Our model is inspired by the classic self-training algorithm. However, as opposed to self-training, Self-Pretraining is threshold-free, it can potentially update its belief about previously labeled documents, and can cope with the semantic drift problem. Self-Pretraining is iterative and consists of two classifiers.In each iteration, one classifier draws a random set of unlabeled documents and labels them. This set is used to initialize the second classifier, to be further trained by the set of labeled documents. The algorithm proceeds to the next iteration and the classifiers' roles are reversed. To improve the flow of information across the iterations and also to cope with the semantic drift problem, Self-Pretraining employs an iterative distillation process, transfers hypotheses across the iterations, utilizes a two-stage training model, uses an efficient learning rate schedule, and employs a pseudo-label transformation heuristic. We have evaluated our model in three publicly available social media datasets. Our experiments show that Self-Pretraining outperforms the existing state-of-the-art semisupervised classifiers across multiple settings. Our code is available at https://github.com/p-karisani/self_pretraining.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Co-Decomp. We included the framework introduced in [30]. This model uses domain knowledge to decompose the task into a set of subtasks to be solved in a multi-view setting.…”

Section: Baselinesmentioning

confidence: 99%

Section: Empirical Analysismentioning

confidence: 99%

See 1 more Smart Citation

Semi-Supervised Text Classification via Self-Pretraining

Karisani

2021

Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…EWC is implemented extensively in different applications such as in multi-task learning (Rebuffi et al, 2018) and mining tasks (Karisani et al, 2020). Unlike the previous application of EWC which focuses on working in different domains, we apply EWC to address the data imbalance problem.…”

Section: Elastic Weight Consolidationmentioning

confidence: 99%

Sequential Targeting: an incremental learning approach for data imbalance in text classification

Jang,

Kim,

Choi

et al. 2020

Preprint

View full text Add to dashboard Cite

Classification tasks require a balanced distribution of data to ensure the learner to be trained to generalize over all classes. In realworld datasets, however, the number of instances vary substantially among classes. This typically leads to a learner that promotes bias towards the majority group due to its dominating property. Therefore, methods to handle imbalanced datasets are crucial for alleviating distributional skews and fully utilizing the under-represented data, especially in text classification. While addressing the imbalance in text data, most methods utilize sampling methods on the numerical representation of the data, which limits its efficiency on how effective the representation is. We propose a novel training method, Sequential Targeting(ST), independent of the effectiveness of the representation method, which enforces an incremental learning setting by splitting the data into mutually exclusive subsets and training the learner adaptively. To address problems that arise within incremental learning, we apply elastic weight consolidation. We demonstrate the effectiveness of our method through experiments on simulated benchmark datasets (IMDB) and data collected from NAVER.

show abstract

“…Modern classifiers typically rely on large amount of training data. Collecting large training sets is expensive and in some cases very challenging, e.g., in legal domain (Holzenberger, Blair-Stanek, and Van Durme 2020) or in social media domain (Karisani, Choi, and Xiong 2021;Karisani and Karisani 2020). There exist several techniques to address the lack of training data, one of which is Domain Adaptation (Ben-David and Schuller 2003), where a classifier is trained in one domain (the source domain) and evaluated in another domain (the target domain).…”

Section: Introductionmentioning

confidence: 99%

Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired Classifiers

Karisani

2022

AAAI

Self Cite

View full text Add to dashboard Cite

We present a novel multiple-source unsupervised model for text classification under domain shift. Our model exploits the update rates in document representations to dynamically integrate domain encoders. It also employs a probabilistic heuristic to infer the error rate in the target domain in order to pair source classifiers. Our heuristic exploits data transformation cost and the classifier accuracy in the target feature space. We have used real world scenarios of Domain Adaptation to evaluate the efficacy of our algorithm. We also used pretrained multi-layer transformers as the document encoder in the experiments to demonstrate whether the improvement achieved by domain adaptation models can be delivered by out-of-the-box language model pretraining. The experiments testify that our model is the top performing approach in this setting.

show abstract

Domain-Guided Task Decomposition with Self-Training for Detecting Personal Events in Social Media

Cited by 13 publications

References 19 publications

Semi-Supervised Text Classification via Self-Pretraining

Semi-Supervised Text Classification via Self-Pretraining

Sequential Targeting: an incremental learning approach for data imbalance in text classification

Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired Classifiers

Contact Info

Product

Resources

About