Transformer-based deep learning models for the sentiment analysis of social media data

Kokab, Sayyida Tabinda; Asghar, Sohail; Naz, Shehneela

doi:10.1016/j.array.2022.100157

Cited by 86 publications

(32 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…En esta investigación nos centramos únicamente en la comparación de dos modelos de Deep Learning, LSTM y Bi-LSTM, ya que nos interesa estudiar el comportamiento de las celdas LSTM y cómo éstas agregan información a largo plazo de una forma secuencial, por ello su popularidad. Sin embargo, hay otros modelos basados en RNNs como los Transformadores, Transformer Learner (TL) en inglés (Tabinda Kokab et al, 2022). Los TL conforman una extensión de las RNN, y pueden realizar muchos cómputos en paralelo, mientras que los modelos LSTM y Bi-LSTM deben computar secuencialmente.…”

Section: Resultados Y Discusiónunclassified

Análisis de sentimientos en Twitter

Lovera¹,

Cardinale²

2023

Rev. Cient. Sist. Inform.

View full text Add to dashboard Cite

El análisis de sentimientos ayuda a determinar la percepción de usuarios en diferentes aspectos de la vida cotidiana, como preferencias de productos en el mercado, nivel de confianza de los usuarios en ambientes de trabajo, o preferencias políticas. La idea es predecir tendencias o preferencias basados en sentimientos. En este artículo evaluamos las técnicas más comunes usadas para este tipo de análisis, considerando técnicas de aprendizaje de máquina y aprendizaje de máquina profundo. Nuestra contribución principal se basa en una propuesta de una estrategia metodológica que abarca las fases de preprocesamiento de datos, construcción de modelos predictivos y su evaluación. De los resultados, el mejor modelo clásico fue SVM, con 78% de precisión, y 79% de métrica F1 (F1 score). Para los modelos de Deep Learning, con mejores resultados fueron los modelos clásicos. El modelo con mejor desempeño fue el de Deep Learning Long Short Term Memory (LSTM), alcanzando un 88% de precisión y 89% de métrica F1. El peor de los modelos de Deep Learning fue el CNN, con 77% de precisión como de métrica F1. Concluyendo que, el algoritmo Long Short Term Memory (LSTM) demostró ser el mejor rendimiento, alcanzando hasta un 89% de precisión.

show abstract

Section: Resultados Y Discusiónunclassified

Análisis de sentimientos en Twitter

Lovera¹,

Cardinale²

2023

Rev. Cient. Sist. Inform.

View full text Add to dashboard Cite

show abstract

“…The choice of PLMs, GPT-2 [40], BERT [15], XLNet [58], and T5 [41], that are utilized in this study to identify affective bias, is motivated by considering their acceptance as relevant and neoteric contextualized models with high performance efficacy towards textual emotion detection [3,4] and the much related task of sentiment analysis [50,59] within the area of affective computing. GPT and BERT are the very popular PLMs that follow the most effective auto-regressive and auto-encoding self-supervised pre-training objectives, respectively, where GPT uses transformer decoder blocks, whereas BERT uses transformer encoder blocks.…”

Section: Textual Emotion Detection Using Large Plmsmentioning

confidence: 99%

Blacks is to Anger as Whites is to Joy? Understanding Latent Affective Bias in Large Pre-trained Neural Language Models

Kadan¹,

Deepak²,

Bhadra³

et al. 2023

Preprint

View full text Add to dashboard Cite

Groundbreaking inventions and highly significant performance improvements in deep learning based Natural Language Processing are witnessed through the development of transformer based large Pre-trained Language Models (PLMs). The wide availability of unlabeled data within human generated data deluge along with self-supervised learning strategy helps to accelerate the success of large PLMs in language generation, language understanding, etc. But at the same time, latent historical bias/unfairness in human minds towards a particular gender, race, etc., encoded unintentionally/intentionally into the corpora harms and questions the utility and efficacy of large PLMs in many real-world applications, particularly for the protected groups. In this paper, we present an extensive investigation towards understanding the existence of "Affective Bias" in large PLMs to unveil any biased association of emotions such as anger, fear, joy, etc., towards a particular gender, race or religion with respect to the downstream task of textual emotion detection. We conduct our exploration of affective bias from the very initial stage of corpus level affective bias analysis by searching for imbalanced distribution of affective words within a domain, in large scale corpora that are used to pre-train and fine-tune PLMs.Later, to quantify affective bias in model predictions, we perform an extensive set of class-based and intensity-based evaluations using various bias evaluation corpora. Our results show the existence of statistically significant affective bias in the PLM based emotion detection systems, indicating biased association of certain emotions towards a particular gender, race, and religion.

show abstract

“…The question is that this dataset might be not big enough to capture the characteristics of the target industry sectors at the parameter turning step when the language models were retrained. • Transformer models had proven to be state-of-the-art architecture when working on some sentence-level NLP tasks such as sentiment analysis [37] and question answering [38]. Sentence-based text sequences facilitate the semantic learning of local contexts for transformer models.…”

Section: Algorithms and Techniques Used In Industry Detectionmentioning

confidence: 99%

Automatic Detection of Industry Sectors in Legal Articles Using Machine Learning Approaches

Ye¹,

Hadjiantoni²,

Long³

et al. 2023

Preprint

View full text Add to dashboard Cite

The ability to automatically identify industry sector coverage in articles on legal developments, or any kind of news articles for that matter, can bring plentiful of benefits both to the readers and the content creators themselves. By having articles tagged based on industry coverage, readers from all around the world would be able to get to legal news that are specific to their region and professional industry. Simultaneously, writers would benefit from understanding which industries potentially lack coverage or which industries readers are currently mostly interested in and thus, they would focus their writing efforts towards more inclusive and relevant legal news coverage. In this paper, a Machine Learning-powered industry analysis approach which combined Natural Language Processing (NLP) with Statistical and Machine Learning (ML)

show abstract

Transformer-based deep learning models for the sentiment analysis of social media data

Cited by 86 publications

References 47 publications

Análisis de sentimientos en Twitter

Análisis de sentimientos en Twitter

Blacks is to Anger as Whites is to Joy? Understanding Latent Affective Bias in Large Pre-trained Neural Language Models

Automatic Detection of Industry Sectors in Legal Articles Using Machine Learning Approaches

Contact Info

Product

Resources

About