EVALITA4ELG: Italian Benchmark Linguistic Resources, NLP Services and Tools for the ELG Platform

Patti, Viviana; Basile, Valerio; Bosco, Cristina; Varvara, Rossella; Fell, Michael; Bolioli, Andrea; Bosca, Alessio

doi:10.4000/ijcol.754

Cited by 4 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One direction for possible future work is to improve the inference speed of the models by either distilling their knowledge in a smaller network (Hinton et al, 2015) or quantizing their weights (Yang et al, 2019). Furthermore, we intend to include our results for legal document classification in language specific NLP benchmarks such as KLEJ for Polish (Rybak et al, 2020), LiRo for Romanian (Dumitrescu et al, 2021) or EVALITA4ELG for Italian (Patti et al, 2020).…”

Section: Discussionmentioning

confidence: 99%

PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors

Avram

Pais

Tufiș

2021

Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing M

View full text Add to dashboard Cite

EuroVoc is a multilingual thesaurus that was built for organizing the legislative documentary of the European Union institutions. It contains thousands of categories at different levels of specificity and its descriptors are targeted by legal texts in almost thirty languages. In this work we propose a unified framework for Eu-roVoc classification on 22 languages by finetuning modern Transformer-based pretrained language models. We study extensively the performance of our trained models and show that they significantly improve the results obtained by a similar tool -JEX -on the same dataset. The code and the fine-tuned models were open sourced, together with a programmatic interface that eases the process of loading the weights of a trained model and of classifying a new document.

show abstract

Section: Discussionmentioning

confidence: 99%

PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors

Avram

Pais

Tufiș

2021

Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing M

View full text Add to dashboard Cite

show abstract

“…While most of the studies regard sentences and documents that are written in the English language [6], there are important works also focusing on Italian [40]. For instance, in [41] a conversational agent for the Italian public administration is presented.…”

Section: Related Workmentioning

confidence: 99%

A Performance Comparison of Different Cloud-Based Natural Language Understanding Services for an Italian e-Learning Platform

Zubani

Sigalini²,

Serina

et al. 2022

Future Internet

View full text Add to dashboard Cite

During the COVID-19 pandemic, the corporate online training sector has increased exponentially and online course providers had to implement innovative solutions to be more efficient and provide a satisfactory service. This paper considers a real case study in implementing a chatbot, which answers frequently asked questions from learners on an Italian e-learning platform that provides workplace safety courses to several business customers. Having to respond quickly to the increase in the courses activated, the company decided to develop a chatbot using a cloud-based service currently available on the market. These services are based on Natural Language Understanding (NLU) engines, which deal with identifying information such as entities and intentions from the sentences provided as input. To integrate a chatbot in an e-learning platform, we studied the performance of the intent recognition task of the major NLU platforms available on the market with an in-depth comparison, using an Italian dataset provided by the owner of the e-learning platform. We focused on intent recognition, carried out several experiments and evaluated performance in terms of F-score, error rate, response time, and robustness of all the services selected. The chatbot is currently in production, therefore we present a description of the system implemented and its results on the original users’ requests.

show abstract

“…The current version of the ontology comprises 148 classes, 37 object properties and nine data properties. The ontology and the KG are thoroughly described in Patti et al (2020). As an example, Figure 1 depicts the structure of the KG around the HaSpeeDe2018 task.…”

Section: The Evalita Knowledge Graphmentioning

confidence: 99%

Italian EVALITA Benchmark Linguistic Resources, NLP Services and Tools

Patti

Basile

Bolioli³

et al. 2022

European Language Grid

View full text Add to dashboard Cite

Starting from the first edition held in 2007, EVALITA is the initiative for the evaluation of Natural Language Processing tools for Italian. We describe the EVALITA4ELG project, whose main aim is to systematically collect the resources released as benchmarks for this evaluation campaign, and make them easily accessible through the European Language Grid platform. The collection is moreover integrated with systems and baselines as a pool of web services with a common interface, deployed on a dedicated hardware infrastructure.

show abstract

EVALITA4ELG: Italian Benchmark Linguistic Resources, NLP Services and Tools for the ELG Platform

Cited by 4 publications

References 39 publications

PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors

PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors

A Performance Comparison of Different Cloud-Based Natural Language Understanding Services for an Italian e-Learning Platform

Italian EVALITA Benchmark Linguistic Resources, NLP Services and Tools

Contact Info

Product

Resources

About