Dependency parsing of learner English

Abstract

Current syntactic annotation of large-scale learner corpora mainly resorts to “standard parsers” trained on native language data. Understanding how these parsers perform on learner data is important for downstream research and application related to learner language. This study evaluates the performance of multiple standard probabilistic parsers on learner English. Our contributions are three-fold. Firstly, we demonstrate that the common practice of constructing a gold standard – by manually correcting the pre-annotation of a single parser – can introduce bias to parser evaluation. We propose an alternative annotation method which can control for the annotation bias. Secondly, we quantify the influence of learner errors on parsing errors, and identify the learner errors that impact on parsing most. Finally, we compare the performance of the parsers on learner English and native English. Our results have useful implications on how to select a standard parser for learner English.

A Supervised Learning Model for the Automatic Assessment of Language Levels Based on Learner Errors

“…Similarly, morphosyntactic error types may be identified by using POS patterns. [14] reports the robustness of parsers when analysing learner data, and that dependency parsing is more sensitive to errors than PoS-tagging. Conversely, error types such as verb tense remain challenging in terms of implementation due to the semantic value of contexts.…”

Using Classifier Features to Determine Language Transfer on Morphemes

“…We collect data from the EF Cambridge Open Language Database (EFCamDat), a longitudinal corpus consisting of 1.8 million English learner texts submitted by 174,000 students enrolled in a virtual learning environment (Huang et al, 2017). The corpus spans sixteen proficiency levels aligned with standards such as the Common European Framework of Reference for languages.…”

“…The texts include metadata on the learner's proficiency level and nationality acts as a proxy to native language background. The texts are annotated with part of speech (POS) tags using the Penn Treebank Tagset, grammatical relationships using Syn-taxNet, and some texts in the corpus (66%) include error annotations provided by teachers using predetermined error markup tags (Huang et al, 2017).…”

Working with Data from Real-World Corpora: A Case Study on Identifying Issues and Using Scalable Solutions

“…The solution: a case study The original database. The EF Cambridge Open Language Database (EFCAMDAT) is the largest open-access L2 English learner database, containing ~1,180,000 texts written by ~174,000 learners from various nationalities (Geertzen et al, 2013;Huang et al, 2017Huang et al, , 2018. It contains data that was gathered from an online educational platform, which was run in many different countries over the course of years.…”