Exploring Reusability and Reproducibility for a Research Infrastructure for L1 and L2 Learner Corpora

Abstract

Up until today research in various educational and linguistic domains such as learner corpus research, writing research, or second language acquisition has produced a substantial amount of research data in the form of L1 and L2 learner corpora. However, the multitude of individual solutions combined with domain-inherent obstacles in data sharing have so far hampered comparability, reusability and reproducibility of data and research results. In this article, we present work in creating a digital infrastructure for L1 and L2 learner corpora and populating it with data collected in the past. We embed our infrastructure efforts in the broader field of infrastructures for scientific research, drawing from technical solutions and frameworks from research data management, among which the FAIR guiding principles for data stewardship. We share our experiences from integrating some L1 and L2 learner corpora from concluded projects into the infrastructure while trying to ensure compliance with the FAIR principles and the standards we established for reproducibility, discussing how far research data that has been collected in the past can be made comparable, reusable and reproducible. Our results show that some basic needs for providing comparable and reusable data are covered by existing general infrastructure solutions and can be exploited for domain-specific infrastructures such as the one presented in this article. Other aspects need genuinely domain-driven approaches. The solutions found for the corpora in the presented infrastructure can only be a preliminary attempt, and further community involvement would be needed to provide templates and models acknowledged and promoted by the community. Furthermore, forward-looking data management would be needed starting from the beginning of new corpus creation projects to ensure that all requirements for FAIR data can be met.

Proceedings of the 12th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2023)

“…Following the 6th International Conference for Learner Corpus Research (LCR 2022), a public call for feedback on a new draft of the core metadata standard has been sent to several relevant mailing lists 7 . Furthemore, at the same conference König et al (2022) presented their approach to testing the core metadata standard on several corpora and expressing it using CMDI.…”

“…For example CMDI is already omnipresent for all data published within CLARIN and can be modified to fit the data using profiles. As show by König et al (2022), it could form a starting point for a unified representation for learner corpora metadata. And because it is a standard format within a large infrastructure, existing tools can be used to create and modify the metadata for learner corpora.…”

On two SweLL learner corpora – SweLL-pilot and SweLL-gold

“…Metadata for learner corpora is extremely important for pursuing different types of research and for the interoperability between corpora [19,20]. For example, age, gender and first languages are important for identification of learning problems for different demographic groups; task metadata -for studying the impact of the task on the type of language produced by learners in the essays.…”

“…Work on metadata standardization in LCR was initiated by Paquot and Granger in 2017 [21], was followed up by König et al in 2022 [22] and is still ongoing [23]. Paquot et al [23] identify eight groups of metadata -administrative, corpus design, learner, text, task, annotation, annotator and transcriber 2 -with multiple subcategories divided into obligatory and optional.…”

Annotation Management Tool: A Requirement for Corpus Construction

“…Among others, we have tested uploading other (bonus) learner corpora to the portal, and exporting them from the portal applying a unified set of metadata attributes and values (using 'N/A' as a value for absent attributes). This step has helped us make several Swedish learner corpora interoperable with each other, interoperability being a known challenge in CLARIN-related context (König et al, 2021;Stemle et al, 2019;Volodina et al, 2018).…”