Exploring Linguistic Acceptability in Swedish Learners’ Language

Abstract

We present our initial experiments on binary classification of sentences into linguistically correct versus incorrect ones in Swedish using the DaLAJ dataset (Volodina et al., 2021a). The nature of the task is bordering on linguistic acceptability judgments, on the one hand, and on grammatical error detection task, on the other. The experiments include models trained with different input features and on different variations of the training, validation, and test splits. We also analyze the results focusing on different error types and errors made on different proficiency levels. Apart from insights into which features and approaches work well for this task, we present first benchmark results on this dataset. The implementation is based on a bidirectional LSTM network and pretrained FastText embeddings, BERT embeddings, own word and character embeddings, as well as part-of-speech tags and dependency labels as input features. The best model used BERT embeddings and a training and validation set enriched with additional correct sentences. It reached an accuracy of 73% on one of three test sets used in the evaluation. These promising results illustrate that the data and format of DaLAJ make a valuable new resource for research in acceptability judgements in Swedish.

On two SweLL learner corpora – SweLL-pilot and SweLL-gold

“…A number of derivative resources have been developed since 2016 based on the two SweLL corpora, such as wordlists for language learners -SweLLex [52] and later Sen*Lex [45] -for studies on lexical competences of L2 learners; DaLAJ [60] for studies on linguistic acceptability [61], CoDeRooMor [62] for studying derivational morphology of Swedish, MuClaGED [63] for error classification, synthetic datasets imitating real-life errors [64] and many others. The Swedish MultiGED dataset 11 based on SweLL-gold has been used for the MultiGED shared task [48] and we plan new shared tasks based on the SweLL corpora in the near future.…”

DaLAJ-GED - a dataset for Grammatical Error Detection tasks on Swedish

“…• advanced: C1 level (C2 missing in Coctaill). This version of DaLAJ is an official improved variant of the previously tested experimental version presented in Klezl et al (2022).…”

Explorando Redes Neurais Profundas para Tarefa de Aceitabilidade Linguística

“…Aceitabilidade linguística é a tarefa de determinar se uma sentenc ¸a está gramaticalmente correta. Essa tarefa vem do campo da linguística generativa [Klezl et al 2022] que se baseia em julgamentos intuitivos de falantes nativos sobre se uma sentenc ¸a é aceitável ou não [T Schütze 2016]. Essa tarefa possui diversas aplicac ¸ões na área de Processamento de Línguas Naturais (PLN), por exemplo: analisar a robustez de modelos de língua [Yin et al 2020] e verificar se tais modelos adquirem conhecimentos gramaticais [Zhang et al 2021, Choshen et al 2022.…”