T-BERT -- Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT

Palani, Sarojadevi; Rajagopal, Prabhu; Pancholi, Sidharth

doi:10.48550/arxiv.2106.01097

Cited by 2 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The layer matrix parameters for the classifier were W ∈ R KxH , where K signifies the category number. The likelihood of every category was calculated using the softmax function, which can be calculated by the probability of each category and presented by the following equations [49].…”

Section: Word Embedding Modelmentioning

confidence: 99%

Multi-Task Aspect-Based Sentiment: A Hybrid Sampling and Stance Detection Approach

Waheeb

2023

Applied Sciences

View full text Add to dashboard Cite

This paper discusses the challenges associated with a class imbalance in medical data and the limitations of current approaches, such as machine multi-task learning (MMTL), in addressing these challenges. The proposed solution involves a novel hybrid data sampling method that combines SMOTE, a meta-weigher with a meta-based self-training method (MMS), and one-sided selection (OSS) to balance the distribution of classes. The method also utilizes condensed nearest neighbors (CNN) to remove noisy majority examples and redundant examples. The proposed technique is twofold, involving the creation of artificial instances using SMOTE-OSS-CNN to oversample the under-represented class distribution and the use of MMS to train an instructor model that produces in-field knowledge for pseudo-labeled examples. The student model uses these pseudo-labels for supervised learning, and the student model and MMS meta-weigher are jointly trained to give each example subtask-specific weights to balance class labels and mitigate the noise effects caused by self-training. The proposed technique is evaluated on a discharge summary dataset against six state-of-the-art approaches, and the results demonstrate that it outperforms these approaches with complete labeled data and achieves results equivalent to state-of-the-art methods that require all labeled data using aspect-based sentiment analysis (ABSA).

show abstract

Section: Word Embedding Modelmentioning

confidence: 99%

Multi-Task Aspect-Based Sentiment: A Hybrid Sampling and Stance Detection Approach

Waheeb

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…A distinct Transformer-based method is T-BERT [12], which enhances performance in sentiment classification by combining latent topics with contextual BERT embedding. T-BERT is a combination of LDA and BERT, which aims to obtain contextual topics on which the authors further apply BERT for sentiment analysis.…”

Section: Introductionmentioning

confidence: 99%

Romanian Topic Modeling – An Evaluation of Probabilistic Versus Transformer-Based Topic Modeling for Domain Categorization

NIȚU,

DASCĂLU,

DASCĂLU

2023

RRST-EE

View full text Add to dashboard Cite

A primary challenge for digital library systems when digitizing millions of volumes is to automatically analyze and group the huge document collection by categories while identifying patterns and extracting the main themes. A common method to be leveraged on unlabeled texts is topic modeling. Given the wide range of datasets and evaluation criteria used by researchers, comparing the performance and outputs of existing unsupervised algorithms is a complex task. This paper introduces a domain-based topic modeling evaluation applied to Romanian documents. Several variants of Latent Dirichlet Allocation (LDA) combined with dimensionality reduction techniques were compared to Transformer-based models for topic modeling. Experiments were conducted on two datasets of varying text lengths: abstracts of novels and full-text documents. Evaluations were performed against coherence and silhouette scores, while the validation considered classification and clustering tasks. Results highlighted meaningful topics extracted from both datasets.

show abstract

T-BERT -- Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT

Cited by 2 publications

References 24 publications

Multi-Task Aspect-Based Sentiment: A Hybrid Sampling and Stance Detection Approach

Multi-Task Aspect-Based Sentiment: A Hybrid Sampling and Stance Detection Approach

Romanian Topic Modeling – An Evaluation of Probabilistic Versus Transformer-Based Topic Modeling for Domain Categorization

Contact Info

Product

Resources

About