PRELEARN @ EVALITA 2020: Overview of the Prerequisite Relation Learning Task for Italian

Alzetta, Chiara; Miaschi, Alessio; Dell’Orletta⋄, Felice; Koceva, Frosina; Torre, Ilaria

doi:10.4000/books.aaccademia.7518

Cited by 6 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We did 10-fold cross validation for the in-domain experiments except with the Italian-BERT 9 , for which we used a stratified split of 30% for validation set. Table 4 shows the experimental results over the training (validation) set for both, in-domain and cross-domain scenarios.…”

Section: Discussion: Ablation Analysismentioning

confidence: 99%

“…In this work, we present our systems to automatically detect prerequisite relations for Italian language in the context of the PRELEARN shared task (Alzetta et al, 2020) at EVALITA 2020 (Basile et al, 2020). The evaluation of submissions considers: (1) in-domain and crossdomain scenarios defined by either the inclusion (in-domain) or exclusion (cross-domain) of the target domain in the training set.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

NLP-CIC @ PRELEARN: Mastering Prerequisites Relations, from Handcrafted Features to Embeddings

Angel¹,

Aroyehun²,

Gelbukh³

2020

EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

View full text Add to dashboard Cite

English. We present our systems and findings for the prerequisite relation learning task (PRELEARN) at EVALITA 2020. The task aims to classify whether a pair of concepts hold a prerequisite relation or not. We model the problem using handcrafted features and embedding representations for in-domain and cross-domain scenarios. Our submissions ranked first place in both scenarios with average F1 score of 0.887 and 0.690 respectively across domains on the test sets. We made our code freely available 1 .

show abstract

Section: Discussion: Ablation Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

NLP-CIC @ PRELEARN: Mastering Prerequisites Relations, from Handcrafted Features to Embeddings

Angel¹,

Aroyehun²,

Gelbukh³

2020

EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

View full text Add to dashboard Cite

show abstract

“…Different from word2vec, fastText operated additional tokenization, which breaks word-level tokens into several n-grams to provide representation for out-of-vocabulary words. In 2020, a shared task on Concept Prerequisite Learning (PRELEARN) was organized during the EVALITA 2020 [16]. It is the first shared task on automatic prerequisite learning between educational concepts in Italian.…”

Section: Related Workmentioning

confidence: 99%

“…With the popularization of deep learning technology, various deep learning models show a trend of coming from behind in the concept prerequisite learning tasks [14][15][16]. It has to be mentioned that transformers have become a prevalent deep learning technology in recent years.…”

Section: Concept Prerequisite Relation Discoveringmentioning

confidence: 99%

“…For concept prerequisite learning, most previous researchers used to define handcraft features to discover concept prerequisite relations [10][11][12][13][18][19][20]. Later, some researchers began to use deep learning models such as LSTM [14], fastText [15], and BERT [16] to define embedding features to do that. In this paper, we deeply compared the capabilities of various transformer models in concept prerequisite learning, and combined embedding features with handcraft features for concept prerequisite relation prediction, achieving high accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Extracting Precedence Relations between Video Lectures in MOOCs

Xiao

Bai

Zhang

2022

Proceedings of the 2022 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Nowadays, the high dropout rate has become a widespread phenomenon in various MOOC platforms. When learning a MOOC, many learners are reluctant to spend time learning from the first video lecture to the last one. If we can recommend a learning path based on learners' individual needs and ignore irrelevant video lectures in the MOOC, it will help them learn more efficiently. The premise of learning path recommendation is to understand the precedence relations between learning resources. In this paper, we propose a novel approach for extracting precedence relations between video lectures in a MOOC. According to "knowledge depth" of concepts, we extract the core concepts from the video captions accurately. Transformer-based models are used to discover concept prerequisite relations, which help us identify the precedence relations between video lectures in MOOCs. Experiments show that the proposed method outperforms the state-of-the-art methods. CCS CONCEPTS• Applied computing → E-learning; • Computing methodologies → Information extraction; Reasoning about belief and knowledge.

show abstract

Annotation Protocol for Textbook Enrichment with Prerequisite Knowledge Graph

Alzetta,

Torre,

Koceva

2023

Tech Know Learn

Self Cite

View full text Add to dashboard Cite

Extracting and formally representing the knowledge embedded in textbooks, such as the concepts explained and the relations between them, can support the provision of advanced knowledge-based services for learning environments and digital libraries. In this paper, we consider a specific type of relation in textbooks referred to as prerequisite relations (PR). PRs represent precedence relations between concepts aimed to provide the reader with the knowledge needed to understand a further concept(s). Their annotation in educational texts produces datasets that can be represented as a graph of concepts connected by PRs. However, building good-quality and reliable datasets of PRs from a textbook is still an open issue, not just for automated annotation methods but even for manual annotation. In turn, the lack of good-quality datasets and well-defined criteria to identify PRs affect the development and validation of automated methods for prerequisite identification. As a contribution to this issue, in this paper, we propose PREAP, a protocol for the annotation of prerequisite relations in textbooks aimed at obtaining reliable annotated data that can be shared, compared, and reused in the research community. PREAP defines a novel textbook-driven annotation method aimed to capture the structure of prerequisites underlying the text. The protocol has been evaluated against baseline methods for manual and automatic annotation. The findings show that PREAP enables the creation of prerequisite knowledge graphs that have higher inter-annotator agreement, accuracy, and alignment with text than the baseline methods. This suggests that the protocol is able to accurately capture the PRs expressed in the text. Furthermore, the findings show that the time required to complete the annotation using PREAP are significantly shorter than with the other manual baseline methods. The paper includes also guidelines for using PREAP in three annotation scenarios, experimentally tested. We also provide example datasets and a user interface that we developed to support prerequisite annotation.

show abstract

PRELEARN @ EVALITA 2020: Overview of the Prerequisite Relation Learning Task for Italian

Cited by 6 publications

References 16 publications

NLP-CIC @ PRELEARN: Mastering Prerequisites Relations, from Handcrafted Features to Embeddings

NLP-CIC @ PRELEARN: Mastering Prerequisites Relations, from Handcrafted Features to Embeddings

Extracting Precedence Relations between Video Lectures in MOOCs

Annotation Protocol for Textbook Enrichment with Prerequisite Knowledge Graph

Contact Info

Product

Resources

About