Natural language processing for learner corpus research

Abstract

The term natural language processing (NLP) refers to the use of computer programs to automatically analyze human language. NLP processes range from the (relatively) simple task of splitting character sequences into words and sentences to much more sophisticated (and challenging) tasks such as converting speech sounds into text and annotating texts for syntactic, semantic, and pragmatic features (among others, see Jurafsky & Manning, 2008 for a survey of common NLP processes; and Meurers & Dickinson, 2017 for specific applications to L2 research). NLP tools of varying complexity have played an important role in the development of corpus linguistics in general and learner corpus research (LCR) in particular. Although relatively simple NLP tools such as concordancers (e.g.,

Shaping Writing Grades

“…These methods have the advantage of retrieving combinations efficiently. However, their accuracy with learner writing has only recently started to be documented, so researchers need to exercise caution in their use (e.g., see Huang et al, 2018; and the special issue on working with learner data edited by Kyle (2021)).…”

A Dependency Treebank of Spoken Second Language English

“…Early studies (e.g., Biber, 1988) focused on the analysis of lexical and lexicogrammatical variation across registers (e.g., different spoken and written language use domains). As the subfield of learner corpus research has grown, taggers and parsers have also been used to investigate how second language learners' linguistic patterns change over time (e.g., Crossley & McNamara, 2014;Kyle et al, 2021) and/or differ across proficiency levels (e.g., Biber et al, 2014;Grant & Ginther, 2000;Paquot, 2018).…”

“…Lexicogrammatical features: A number of studies have investigated the relationship between L2 proficiency and the use of lexicogrammatical features that are common in academic writing such as various types of noun phrase elaboration (e.g., Biber et al, 2014;Grant & Ginther, 2000;Picoral et al, 2021). A related line of research has explored the relationship between characteristics of verb argument construction use and L2 writing proficiency (e.g., Kyle & Crossley, 2017;Kyle et al, 2021). Syntactic complexity: A particularly common use of NLP tools in second language research is the calculation of classic syntactic complexity indices such as mean length of clause and dependent clauses per clause (e.g., Lu, 2010Lu, , 2011 or more fine-grained indices such as the number of dependents per nominal (e.g., Díez-Bedmar & Pérez-Paredes, 2020).…”

“…Although taggers and parsers are increasingly accurate (achieving F1 scores of around .98 for POS taggers and .95 for dependency annotation) when evaluated on in-domain texts (i.e., texts with similar linguistic characteristics as the training data), accuracy can drop precipitously for out of domain texts (e.g., McClosky et al, 2006). A pressing issue, then, is the availability of appropriate annotated corpora to test and train tagging and parsing models on the types of data applied linguists often use (Kyle, 2021;Meurers & Dickinson, 2017). Although a treebank of written second language (L2) English is available (Berzak et al, 2016), to our knowledge no treebanks of spoken L2 speech are publicly and freely available.…”

Computing Numerical Images of Student Writing

“…The notion of affordance has been sporadically used in research literature to refer to the general idea of 'potentials' in research tools. Kyle (2021), for instance, uses the term to investigate the potentials of specific NLP tools for learner corpus research, while Dobson (2019) uses the term in a somewhat wider sense to critically examine possibilities and limitations of computational methods in the humanities and social sciences at large. However, if used in a too general sense an important element of affordances get lost, namely that affordances are not generic properties but are relative to agent and environment.…”