MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information

Zhang, Yu; Garg, Shweta; Yu, Meng; Chen, Xiusi

doi:10.1145/3488560.3498384

Cited by 11 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TopClus may also be extended to perform hierarchical topic discovery, perhaps via top-down clustering in the latent space. Other related tasks like taxonomy construction [30] and weaklysupervised text classification [29,41,42,45,68] may benefit from the coherent and distinctive topics generated by TopClus.…”

Section: Discussionmentioning

confidence: 99%

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

Meng,

Zhang,

Huang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Topic models have been the prominent tools for automatic topic discovery from text corpora. Despite their effectiveness, topic models suffer from several limitations including the inability of modeling word ordering information in documents, the difficulty of incorporating external linguistic knowledge, and the lack of both accurate and efficient inference methods for approximating the intractable posterior. Recently, pretrained language models (PLMs) have brought astonishing performance improvements to a wide variety of tasks due to their superior representations of text. Interestingly, there have not been standard approaches to deploy PLMs for topic discovery as better alternatives to topic models. In this paper, we begin by analyzing the challenges of using PLM representations for topic discovery, and then propose a joint latent space learning and clustering framework built upon PLM embeddings. In the latent space, topic-word and document-topic distributions are jointly modeled so that the discovered topics can be interpreted by coherent and distinctive terms and meanwhile serve as meaningful summaries of the documents. Our model effectively leverages the strong representation power and superb linguistic features brought by PLMs for topic discovery, and is conceptually simpler than topic models. On two benchmark datasets in different domains, our model generates significantly more coherent and diverse topics than strong topic models, and offers better topic-wise document representations, based on both automatic and human evaluations. 1 CCS CONCEPTS• Information systems → Clustering; Document topic models; • Computing methodologies → Natural language processing.

show abstract

Section: Discussionmentioning

confidence: 99%

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

Meng,

Zhang,

Huang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Subsequently, topic-model based methods emerged [4,13,14,33,34], which inferred category-aware topics from a limited set of seed words. In the last few years, neural methods has gained prominance [22,23,31,36,39]. They trained neural classifiers using pseudo labels of texts, often relying on generated pseudo-texts or PLMs to detect category-indicative keywords.…”

Section: Related Work 21 Weakly Supervised Text Classificationmentioning

confidence: 99%

“…It is significant and challenging to classify these texts into predefined categories, especially when up-to-date labeled data are hard to access due to the dynamic and open nature of the Web. Consequently, there has been a growing interest in weakly supervised text classification (WSTC) [16,23,31,32,39,40], also known as zero-shot or dataless text classification [3, 4, 13, 14, 22-24, 29, 30, 33, 34, 36, 41], which only requires a limited set of seed words (label names) for each category.…”

Section: Introductionmentioning

confidence: 99%

RulePrompt: Weakly Supervised Text Classification with Prompting PLMs and Self-Iterative Logical Rules

Li,

Zhu,

Wang

et al. 2024

Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Weakly supervised text classification (WSTC), also called zero-shot or dataless text classification, has attracted increasing attention due to its applicability in classifying a mass of texts within the dynamic and open Web environment, since it requires only a limited set of seed words (label names) for each category instead of labeled data. With the help of recently popular prompting Pre-trained Language Models (PLMs), many studies leveraged manually crafted and/or automatically identified verbalizers to estimate the likelihood of categories, but they failed to differentiate the effects of these category-indicative words, let alone capture their correlations and realize adaptive adjustments according to the unlabeled corpus. In this paper, in order to let the PLM effectively understand each category, we at first propose a novel form of rule-based knowledge using logical expressions to characterize the meanings of categories. Then, we develop a prompting PLM-based approach named RulePrompt for the WSTC task, consisting of a rule mining module and a rule-enhanced pseudo label generation module, plus a self-supervised fine-tuning module to make the PLM align with this task. Within this framework, the inaccurate pseudo labels assigned to texts and the imprecise logical rules associated with categories mutually enhance each other in an alternative manner. That establishes a self-iterative closed loop of knowledge (rule) acquisition and utilization, with seed words serving as the starting point. Extensive experiments validate the effectiveness and robustness of our approach, which markedly outperforms state-of-the-art weakly supervised methods. What is more, our approach yields interpretable category rules, proving its advantage in disambiguating easily-confused categories.

show abstract

“…After the t-th iteration, Amazon review belongs to one or more product categories. We use the subset sampled by Zhang et al (2020Zhang et al ( , 2022, which contains 10 categories and 100K reviews. (3) Twitter (Zhang et al, 2017) 6 is a crawl of geo-tagged tweets in New York City from August 2014 to November 2014.…”

Section: Overall Frameworkmentioning

confidence: 99%

Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds

Meng¹,

Wang²,

Wang³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Discovering latent topics from text corpora has been studied for decades. Many existing topic models adopt a fully unsupervised setting, and their discovered topics may not cater to users' particular interests due to their inability of leveraging user guidance. Although there exist seed-guided topic discovery approaches that leverage user-provided seeds to discover topicrepresentative terms, they are less concerned with two factors: (1) the existence of out-ofvocabulary seeds and (2) the power of pretrained language models (PLMs). In this paper, we generalize the task of seed-guided topic discovery to allow out-of-vocabulary seeds. We propose a novel framework, named SEE-TOPIC, wherein the general knowledge of PLMs and the local semantics learned from the input corpus can mutually benefit each other. Experiments on three real datasets from different domains demonstrate the effectiveness of SEETOPIC in terms of topic coherence, accuracy, and diversity. 1

show abstract

MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information

Cited by 11 publications

References 36 publications

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

RulePrompt: Weakly Supervised Text Classification with Prompting PLMs and Self-Iterative Logical Rules

Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds

Contact Info

Product

Resources

About