Discovering Interpretable Topics by Leveraging Common Sense Knowledge

Harrando, Ismail; Troncy, Raphaël

doi:10.1145/3460210.3493586

Cited by 3 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I Harrando et al 2021. [16] Proposed a model which extracts topics from text documents based on the common-sense knowledge available in ConceptNet, combined both ConceptNet's knowledge graph and graph embeddings, specified the importance of modeling choices and criteria for designing the model, and demonstrated that it can be used to label data for a supervised classifier…”

Section: Literature Reviewmentioning

confidence: 99%

Enhancing Topic Prediction Using Machine Learning Techniques and ConceptNet-based Cosine Similarity Measure

Kaur

Singh

Nandi

et al. 2023

Preprint

View full text Add to dashboard Cite

Topic modelling is one of the important research area of Natural Language Processing. Researchers have tried topic modelling for classification, categorization etc., and even the methods of topic modelling are used for sentiment analysis and other areas of language processing. In this paper, a novel method based on cosine similarity is used for topic modelling using Machine Learning Techniques. Cosine similarity measure on synonyms of topics present in the BBC News and Yahoo datasets from Concept Net is a pioneering approach. It has been observed that machine learning techniques applied here enhance accuracy compared to previously applied methods on these two datasets. On Yahoo datasets, the proposed technique increases the accuracy by 8% even over deep learning methods, which shows the efficiency and applicability of the research. Uses of ConceptNet is a novel idea over here, and the combination of ConceptNet with machine learning made the effort successful in comparison to existing state-of-the-art research on topic modelling.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Enhancing Topic Prediction Using Machine Learning Techniques and ConceptNet-based Cosine Similarity Measure

Kaur

Singh

Nandi

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…a: X-Class, (Wang et al, 2021) a SOTA keyword-based method. b: (Harrando and Troncy, 2021). c: Crowdsourced human performance from Alex et al (2021) (they used a selected portion of Banking77).…”

Section: I2 Additional Implementation Detailsmentioning

confidence: 99%

Beyond prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

Yu¹,

Zhao²,

Nie³

et al. 2022

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Recent work has demonstrated that pre-trained language models (PLMs) are zero-shot learners. However, most existing zero-shot methods involve heavy human engineering or complicated self-training pipelines, hindering their application to new situations. In this work, we show that zero-shot text classification can be improved simply by clustering texts in the embedding spaces of PLMs. Specifically, we fit the unlabeled texts with a Bayesian Gaussian Mixture Model after initializing cluster positions and shapes using class names. Despite its simplicity, this approach achieves superior or comparable performance on both topic and sentiment classification datasets and outperforms prior works significantly on unbalanced datasets. We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes. Our approach achieves an average of 20% absolute improvement over prompt-based zero-shot learning. Finally, we compare different PLM embedding spaces and find that texts are well-clustered by topics even if the PLM is not explicitly pre-trained to generate meaningful sentence embeddings. This work indicates that PLM embeddings can categorize texts without task-specific fine-tuning, thus providing a new way to analyze and utilize their knowledge and zero-shot learning ability 1 .

show abstract

“…In this scenario, augmenting the labels can be useful: building a context (manually or automatically) that can be used to refine the quality of the final embedding helps the process of the semantic alignment of labels and texts [8,9]. This augmentation can be carried out in various fashions, e.g., scraping Wikipedia descriptions, exploring the WordNet taxonomy, or navigating a CommonSense graph [10] to gather related content, words and meaningful relations that could help maximise the similarity between a context and a given example.…”

Section: Related Workmentioning

confidence: 99%

Zero-Shot Topic Labeling for Hazard Classification

2022

View full text Add to dashboard Cite

Topic classification is the task of mapping text onto a set of meaningful labels known beforehand. This scenario is very common both in academia and industry whenever there is the need of categorizing a big corpus of documents according to set custom labels. The standard supervised approach, however, requires thousands of documents to be manually labelled, and additional effort every time the label taxonomy changes. To obviate these downsides, we investigated the application of a zero-shot approach to topic classification. In this setting, a subset of these topics, or even all of them, is not seen at training time, challenging the model to classify corresponding examples using additional information. We first show how zero-shot classification can perform the topic-classification task without any supervision. Secondly, we build a novel hazard-detection dataset by manually selecting tweets gathered by LINKS Foundation for this task, where we demonstrate the effectivenes of our cost-free method on a real-world problem. The idea is to leverage a pre-trained text-embedder (MPNet) to map both text and topics into the same semantic vector space where they can be compared. We demonstrate that these semantic spaces are better aligned when their dimension is reduced, keeping only the most useful information. We investigated three different dimensionality reduction techniques, namely, linear projection, autoencoding and PCA. Using the macro F1-score as the standard metric, it was found that PCA is the best performing technique, recording improvements for each dataset in comparison with the performance on the baseline.

show abstract

Discovering Interpretable Topics by Leveraging Common Sense Knowledge

Cited by 3 publications

References 9 publications

Enhancing Topic Prediction Using Machine Learning Techniques and ConceptNet-based Cosine Similarity Measure

Enhancing Topic Prediction Using Machine Learning Techniques and ConceptNet-based Cosine Similarity Measure

Beyond prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

Zero-Shot Topic Labeling for Hazard Classification

Contact Info

Product

Resources

About