A Multivalued Emotion Lexicon Created and Evaluated by the Crowd

Haralabopoulos, Giannis; Wagner, Christian; McAuley, Derek; Simperl, Elena

doi:10.1109/snams.2018.8554956

Cited by 7 publications

(8 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All the models presented, and in extent their ensembles, can be further improved by a range of techniques. Test augmentation [70], hyperparameter optimization [71], bias reduction [72] and tailored emotional embeddings [4,73] are some techniques that could further improve the generalization capabilities of our networks. However, the computational load over multiple iterations is extensive, as the most complex models required hours of training per epoch and dataset.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Ensemble Deep Learning for Multilabel Binary Classification of User-Generated Content

2020

Self Cite

View full text Add to dashboard Cite

Sentiment analysis usually refers to the analysis of human-generated content via a polarity filter. Affective computing deals with the exact emotions conveyed through information. Emotional information most frequently cannot be accurately described by a single emotion class. Multilabel classifiers can categorize human-generated content in multiple emotional classes. Ensemble learning can improve the statistical, computational and representation aspects of such classifiers. We present a baseline stacked ensemble and propose a weighted ensemble. Our proposed weighted ensemble can use multiple classifiers to improve classification results without hyperparameter tuning or data overfitting. We evaluate our ensemble models with two datasets. The first dataset is from Semeval2018-Task 1 and contains almost 7000 Tweets, labeled with 11 sentiment classes. The second dataset is the Toxic Comment Dataset with more than 150,000 comments, labeled with six different levels of abuse or harassment. Our results suggest that ensemble learning improves classification results by 1.5 % to 5.4 % .

show abstract

Section: Resultsmentioning

confidence: 99%

“…Sentiment analysis is the process by which we uncover sentiment from information. The sentiment part could refer to polarity [1], fine grained or not [2], or to pure emotion information [3][4][5]. The most common source of information for sentiment analysis is Online Social Networks (OSNs) [6,7].…”

Section: Introductionmentioning

confidence: 99%

Ensemble Deep Learning for Multilabel Binary Classification of User-Generated Content

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…During the analysis of the crowdsourcing results from one of our recent studies [9,12], we found evidence of high percentage of spam or dishonest contribution from workers originating from low income countries. Participants that deliver systematically the same contributions were identified as dishonest contributors, with high certainty.…”

Section: Introductionmentioning

confidence: 88%

Paid Crowdsourcing, Low Income Contributors, and Subjectivity

Haralabopoulos

Wagner

McAuley

et al. 2019

IFIP Advances in Information and Communication Technology

Self Cite

View full text Add to dashboard Cite

Scientific projects that require human computation often resort to crowdsourcing. Interested individuals can contribute to a crowdsourcing task, essentially contributing towards the project's goals. To motivate participation and engagement, scientists use a variety of reward mechanisms. The most common motivation, and the one that yields the fastest results, is monetary rewards. By using monetary, scientists address a wider audience to participate in the task. As the payment is below minimum wage for developed economies, users from developing countries are more eager to participate. In subjective tasks, or tasks that cannot be validated through a right or wrong type of validation, monetary incentives could contrast with the much needed quality of submissions. We perform a subjective crowdsourcing task, emotion annotation, and compare the quality of the answers from contributors of varying income levels, based on the Gross Domestic Product. The results indicate a different contribution process between contributors from varying GDP regions. Low income contributors, possibly driven by the monetary incentive, submit low quality answers at a higher pace, while high income contributors provide diverse answers at a slower pace.

show abstract

“…In our study, each word is represented by the emotion it conveys. The Pure Emotion Lexicon (PEL) [11,7] contains a beyond polarity emotion vector, instead of a single emotion (MPQA, WordNet). The emotional vectors are normalised emotion classification results for each term and correspond to the eight basic emotions, as defined by Plutchik [18].…”

Section: Proposed Methodologymentioning

confidence: 99%

Privacy-Preserving Text Labelling Through Crowdsourcing

Haralabopoulos

Torres

Anagnostopoulos

et al. 2021

Artificial Intelligence Applications and Innovations. AIAI 2021 IFIP WG 12.5 International Workshops

Self Cite

View full text Add to dashboard Cite

The extensive use of online social media has highlighted the importance of privacy in the digital space. As more scientists analyse the data created in these platforms, privacy concerns have extended to data usage within the academia. Although text analysis is a well documented topic in academic literature with a multitude of applications, ensuring privacy of user-generated content has been overlooked. In an effort to reduce the exposure of online users' information, we propose a privacy-preserving text labelling method for varying applications, based in crowdsourcing. We transform text with different levels of privacy and analyse the effectiveness of the transformation with regards to label correlation. To demonstrate the adaptive nature of our approach we also employ a TF/IDF filtering transformation. Our results suggest that total privacy can be implemented in labelling, retaining the annotational diversity and subjectivity of traditional labelling. The privacy-preserving labelling, with the use of NRC lexicon, demonstrates an average 0.11 Mean Spearman's Rho correlation, boosted to 0.124 with TF/IDF filtering.

show abstract

A Multivalued Emotion Lexicon Created and Evaluated by the Crowd

Cited by 7 publications

References 46 publications

Ensemble Deep Learning for Multilabel Binary Classification of User-Generated Content

Ensemble Deep Learning for Multilabel Binary Classification of User-Generated Content

Paid Crowdsourcing, Low Income Contributors, and Subjectivity

Privacy-Preserving Text Labelling Through Crowdsourcing

Contact Info

Product

Resources

About