MATILDA - Multi-AnnoTator multi-language InteractiveLight-weight Dialogue Annotator

Cucurnia, Davide; Rozanov, Nikolai; Sucameli, Irene; Ciuffoletti, Augusto; Simi, Maria

doi:10.18653/v1/2021.eacl-demos.5

Cited by 2 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5 However, these tools hide their full functionality behind a paywall. In contrast to the mentioned commercial system, several open-source annotation tools have appeared recently, such as Label Sleuth (Shnarch et al, 2022), Label Studio, 6 , INCEpTION (Klie et al, 2018), MATILDA (Cucurnia et al, 2021), and Paladin (Nghiem et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

ALANNO: An Active Learning Annotation System for Mortals

Jukić,

Jelenić,

Bićanić

et al. 2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrati

View full text Add to dashboard Cite

Supervised machine learning has become the cornerstone of today's data-driven society, increasing the need for labeled data. However, the process of acquiring labels is often expensive and tedious. One possible remedy is to use active learning (AL) -a special family of machine learning algorithms designed to reduce labeling costs. Although AL has been successful in practice, a number of practical challenges hinder its effectiveness and are often overlooked in existing AL annotation tools. To address these challenges, we developed ALANNO, an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects. ALANNO facilitates annotation management in a multi-annotator setup and supports a variety of AL methods and underlying models, which are easily configurable and extensible.

show abstract

Section: Related Workmentioning

confidence: 99%

ALANNO: An Active Learning Annotation System for Mortals

Jukić,

Jelenić,

Bićanić

et al. 2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrati

View full text Add to dashboard Cite

show abstract

“…JILDA and MTurk annotation process was supported by MATILDA, an open source tool specifically designed to annotate multi-turn dialogues, which was extended to support the management of collaborative annotation projects (Cucurnia et al 2021). Each annotator is assigned subsets of the collection to annotate and can add/modify her own annotations without affecting the work of the others.…”

Section: Annotating Jildamentioning

confidence: 99%

Toward Data-Driven Collaborative Dialogue Systems: The JILDA Dataset

Sucameli¹,

Lenci²,

Magnini³

et al. 2021

ijcol

View full text Add to dashboard Cite

Today's goal-oriented dialogue systems are designed to operate in restricted domains and with the implicit assumption that the user goals fit the domain ontology of the system. Under these assumptions dialogues exhibit only limited collaborative phenomena. However, this is not necessarily true in more complex scenarios, where user and system need to collaborate to align their knowledge of the domain in order to improve the conversation and achieve their goals.To foster research on data-driven collaborative dialogues, in this paper we present JILDA, a fully annotated dataset of chat-based, mixed-initiative Italian dialogues related to the job-offer domain. As far as we know, JILDA is the first dialogic corpus completely annotated in this domain. The analysis realised on top of the semantic annotations clearly shows the naturalness and greater complexity of JILDA's dialogues. In fact, the new dataset offers a large number of examples of pragmatic phenomena, such as proactivity (i.e., providing information not explicitly requested) and grounding, which are rarely investigated in AI conversational agents based on neural architectures. In conclusion, the annotated JILDA corpus, given its innovative characteristics, represents a new challenge for conversational agents and an important resource for tackling more complex scenarios, thus advancing the state of the art in this field.

show abstract

MATILDA - Multi-AnnoTator multi-language InteractiveLight-weight Dialogue Annotator

Cited by 2 publications

References 7 publications

ALANNO: An Active Learning Annotation System for Mortals

ALANNO: An Active Learning Annotation System for Mortals

Toward Data-Driven Collaborative Dialogue Systems: The JILDA Dataset

Contact Info

Product

Resources

About