TREMoLo-Tweets: a Multi-Label Corpus of French Tweets for Language Register Characterization

Mekki, Jade; Lecorvé, Gwénolé; Battistelli, Delphine; Béchet, Nicolas

doi:10.26615/978-954-452-072-4_108

Cited by 1 publication

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, due to the larger number of annotators and output labels, as well as low IAA, the individual degrees of belonging for specific labels in our dataset become significantly small. As a result, the thresholding approach (>0.5) employed in [70] for converting degrees of belonging to one-hot encoded vectors is inadequate for emotions in our case. To address this, we adopt an alternative approach by selecting the top N emotion labels with the highest degrees of belonging.…”

Section: Post-processing Annotationsmentioning

confidence: 97%

“…For example, two annotators assigning rank 1 to a text is not equivalent to one assigning rank 1 and the other assigning rank 2 or 3. This problem is addressed in [70] by introducing an approach that transforms ranked annotations from multiple annotators into normalized degrees of class membership. These values range from 0 to 1 and sum up to 1, where higher values correspond to a more pronounced presence of emotion.…”

Section: Post-processing Annotationsmentioning

confidence: 99%

“…For the selected tweets, the normalized degrees of class membership are calculated to derive the final aggregated values. The approach was evaluated on the TREMoLO dataset [70], which contains tweets annotated with language registers indicating whether a tweet is formal, neutral, or casual. These label annotations are less subjective compared to emotion classification, resulting in a less complex problem setting.…”

Section: Post-processing Annotationsmentioning

confidence: 99%

Section: Post-processing Annotationsmentioning

confidence: 99%

“…First, ranks are transformed into degrees of belonging using the formula in [70]. For the sake of brevity, we do not give the formula here, but refer the reader to [70] for the details of this calculation. However, tweets are not subsequently filtered using the voting mechanism, since it results in the removal of a large number of tweets in our dataset due to the low IAA.…”

Section: Post-processing Annotationsmentioning

confidence: 99%

See 4 more Smart Citations

Pseudo-Labeling With Large Language Models for Multi-Label Emotion Classification of French Tweets

Malik,

Bernard,

Pauchet

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This study proposes a novel semi-supervised multi-label emotion classification approach for French tweets based on pseudo-labeling. Human subjectivity in emotional expression makes it difficult for a machine to learn. Therefore, it necessitates training supervised learning models on large datasets annotated by multiple annotators. However, creating such datasets can be costly and time-consuming. Moreover, aggregating annotations from multiple annotators to capture their collective emotional state adds complexity to the task. Semi-supervised learning techniques have shown effectiveness with limited datasets. Furthermore, Large language Models (LLMs), particularly Chat-GPT, have demonstrated superior annotation accuracy compared to annotations obtained from crowdsourcing platforms, when both are evaluated against expertannotated data. This work introduces a novel approach for multi-label emotion classification of French tweets by leveraging pseudo-labels generated through Chat-GPT, a robust large language model. Using zero-shot, one-shot, and few-shot learning techniques, Chat-GPT annotates the unlabelled part of our dataset. These Chat-GPTannotated pseudo-labels are then merged with manual annotations, facilitating the training of a multilabel emotion classification model via semi-supervised learning. Furthermore, within the context of our research, we present a new French tweet dataset, containing testimonials from people affected by an urban industrial incident. This dataset features 2,350 tweets, each manually annotated by three human annotators based on 8 pre-identified emotions. Benchmark results are presented for multi-label emotion classification models employing both fully supervised and semi-supervised approaches with pseudo-labeling. Our findings demonstrate the superiority of our approach for multi-label emotion classification over standard pseudolabeling and an established baseline.

show abstract

Section: Post-processing Annotationsmentioning

confidence: 97%

Section: Post-processing Annotationsmentioning

confidence: 99%

Section: Post-processing Annotationsmentioning

confidence: 99%

Section: Post-processing Annotationsmentioning

confidence: 99%

Section: Post-processing Annotationsmentioning

confidence: 99%

See 3 more Smart Citations