Evalita 2018: Overview on the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian

Caselli, Tommaso; Novielli, Nicole; Patti, Viviana; Rosso, Paolo

doi:10.4000/books.aaccademia.4437

Cited by 27 publications

(18 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…9 https://huggingface.co/m-polignano-uniba/ bert_uncased_L-12_H-768_A-12_italian_alb3rt0 (Barbieri et al 2016) carried out at EVALITA 2016 (Basile et al 2016), a challenge on sentiment analysis on Italian tweets. The second is the hate speechannotated corpus released for the HaSpeeDe shared task (Bosco et al 2018) held at EVALITA 2018 (Caselli et al 2018), a challenge on the detection of hateful content in Italian social media. We verified that the texts contained in those datasets come from a distribution different from the ones used for the pre-training of AlBERTo.…”

Section: Evaluation Of Alberto On Nlp Tasksmentioning

confidence: 99%

AlBERTo: Modeling Italian Social Media Language with BERT

Polignano¹,

Basile²,

Basile³

et al. 2019

ijcol

View full text Add to dashboard Cite

Natural Language Processing tasks recently achieved considerable interest and progresses following the development of numerous innovative artificial intelligence models released in recent years. The increase in available computing power has made possible the application of machine learning approaches on a considerable amount of textual data, demonstrating how they can obtain very encouraging results in challenging NLP tasks by generalizing the properties of natural language directly from the data. Models such as ELMo, GPT/GPT-2, BERT, ERNIE, and RoBERTa have proved to be extremely useful in NLP tasks such as entailment, sentiment analysis, and question answering. The availability of these resources mainly in the English language motivated us towards the realization of AlBERTo, a natural language model based on BERT and trained on the Italian language. We decided to train AlBERTo from scratch on social network language, Twitter in particular, because many of the classic tasks of content analysis are oriented to data extracted from the digital sphere of users. The model was distributed to the community through a repository on GitHub and the Transformers library (Wolf et al. 2019) released by the development group huggingface.co. We have evaluated the validity of the model on the classification tasks of sentiment polarity, irony, subjectivity, and hate speech. The specifications of the model, the code developed for training and fine-tuning, and the instructions for using it in a research project are freely available.

show abstract

Section: Evaluation Of Alberto On Nlp Tasksmentioning

confidence: 99%

AlBERTo: Modeling Italian Social Media Language with BERT

Polignano¹,

Basile²,

Basile³

et al. 2019

ijcol

View full text Add to dashboard Cite

show abstract

“…For English and Spanish, we adopt the data sets proposed in the shared task of hate speech against immigrants and women on Twitter (HatEval) (Basile et al, 2019). For Italian, we consider two different corpora proposed for Evalita shared tasks (Caselli et al, 2018): the automatic misogyny identification challenge (AMI) (Fersini et al, 2018) for hate speech towards women, and the hate speech detection shared task on Facebook and Twitter (HaSpeeDe) (Bosco et al, 2018) for hate speech towards immigrants.…”

Section: Introductionmentioning

confidence: 99%

Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection

Nozza¹

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Reducing and counter-acting hate speech on Social Media is a significant concern. Most of the proposed automatic methods are conducted exclusively on English and very few consistently labeled, non-English resources have been proposed. Learning to detect hate speech on English and transferring to unseen languages seems an immediate solution. This work is the first to shed light on the limits of this zero-shot, cross-lingual transfer learning framework for hate speech detection. We use benchmark data sets in English, Italian, and Spanish to detect hate speech towards immigrants and women. Investigating post-hoc explanations of the model, we discover that nonhateful, language-specific taboo interjections are misinterpreted as signals of hate speech. Our findings demonstrate that zero-shot, crosslingual models cannot be used as they are, but need to be carefully designed.

show abstract

“…The task is part of EVALITA 2018, the periodic evaluation campaign of Natural Language Processing (NLP) and speech tools for the Italian language (Caselli et al, 2018).…”

Section: Motivationmentioning

confidence: 99%

Ghigliottin-AI @ EVALITA2020: Evaluating Artificial Players for the Language Game “La Ghigliottina”

Basile¹,

Lovetere²,

Monti³

et al. 2020

EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

View full text Add to dashboard Cite

English. This paper describes the first edition of the "Solving language games" (NLP4FUN) task at the EVALITA 2018 campaign. The task consists in designing an artificial player for "The Guillotine" (La Ghigliottina, in Italian), a challenging language game which demands knowledge covering a broad range of topics. The game consists in finding a word which is semantically correlated with a set of 5 words called clues. Artificial players for that game can take advantage from the availability of open repositories on the web, such as Wikipedia, that provide the system with the cultural and linguistic background needed to find the solution.

show abstract

Evalita 2018: Overview on the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian

Cited by 27 publications

References 7 publications

AlBERTo: Modeling Italian Social Media Language with BERT

AlBERTo: Modeling Italian Social Media Language with BERT

Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection

Ghigliottin-AI @ EVALITA2020: Evaluating Artificial Players for the Language Game “La Ghigliottina”

Contact Info

Product

Resources

About