Fine-grained Event Classification in News-like Text Snippets - Shared Task 2, CASE 2021

Abstract

This paper describes the Shared Task on Fine-grained Event Classification in News-like Text Snippets. The Shared Task is divided into three subtasks: (a) classification of text snippets reporting socio-political events (25 classes) for which vast amount of training data exists, although exhibiting different structure and style vis-a-vis test data, (b) enhancement to a generalized zero-shot learning problem, where 3 additional event types were introduced in advance, but without any training data ('unseen' classes), and (c) further extension, which introduced 2 additional event types, announced shortly prior to the evaluation phase. The reported Shared Task focuses on classification of events in English texts and is organized as part of the Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021), co-located with the ACL-IJCNLP 2021 Conference. Four teams participated in the task. Best performing systems for the three aforementioned subtasks achieved 83.9%, 79.7% and 77.1% weighted F 1 scores respectively.

AI - Limits and Prospects of Artificial Intelligence

“…Let us take a look at an example, namely the Socio-political and Crisis Events Detection Shared Task at the CASE@ACL-IJCNLP 2021 workshop (Haneczok et al 2021). The task is about the categorization of texts.…”

Template-based Approach to Zero-shot Intent Recognition

“…Zero-shot learning has shown tremendous progress in NLP in recent years. The scope of the tasks, studied in GZS setup, ranges from text classification (Yin et al, 2019) to event extraction (Haneczok et al, 2021;Lyu et al), named entity recognition and entity linking (Logeswaran et al, 2019). A number of datasets for benchmarking zero-shot methods has been developed.…”

“…The GZL classification becomes a binary problem: to predict whether the hypothesis entails the premise or not. Entailment-based approaches have been successfully used for information extraction (Haneczok et al, 2021;Lyu et al;Sainz and Rigau, 2021) and for dataless classification (Ma et al, 2021). However, the entailment-based setup has not been properly explored for GZS intent recognition to the best of our knowledge.…”