Corpora and Language Assessment: The State of the Art

Abstract

This article outlines the current state of and recent developments in the use of corpora for language assessment and considers future directions with a special focus on computational methodology. Because corpora began to make inroads into language assessment in the 1990s, test developers have increasingly used them as a reference resource to become well versed in terms of the linguistic characteristics of expert and novice speakers' usage and identify the test construct. In regard to developing and validating language tests, large representative corpora, learner corpora, and specialized corpora have been actively used, as these corpora have made it possible to systematically compare the linguistic features associated with expert users with those found in learner language. Recent advances in computational approaches to assessment can facilitate this comparison to a great extent using technologies in automated essay scoring and learner language analysis. As an emerging area in the field of language assessment, corpus-based research should extend to less explored areas including compilation and longitudinal analysis of developmental corpora, fine-grained microanalysis of learner's development, and assessment attuned to individual learners who use different linguistic varieties. INTRODUCTIONThis article outlines the current state of and recent developments in the use of corpora for language assessment and considers future directions with a special focus on computational approaches to assessment. A corpus is a principled collection of language materials, spoken or written, compiled into an electronic database for the purpose of linguistic analysis (Biber, Conrad, & Reppen, 1998;Sinclair, 2004). Early discussions on the use of corpora for the purpose of assessing language use dates back to the mid-1990s when the potential of corpora for assessment was first recognized (e.g., Alderson, 1996). Since then, corpora have been increasingly used for large-scale standardized tests as well as for more holistic and developmental assessments. The present article has two related purposes: (a) to review the use of corpora in language assessment and (b) the potential contribution of corpora to language assessment, which include exploratory endeavors drawing on longitudinal, qualitative, and computational techniques. In the following sections, this article offers a survey of related research areas that are beginning to make a mark on the field and computational approaches to assessment drawing on natural language processing (NLP) technology.Correspondence should be sent to Kwanghyun Park,

Does EAP affect written L2 academic stance? A longitudinal learner corpus study

“…This study has shown that for the analysis of learner language, even small longitudinal corpora can be useful for the derivation of 'successful' student texts if performance is measured against a set of criterion used for the assessment of such success. Here, though, we agree with Park (2014) in claiming that existing criterial features used for the assessment of student performance may be unreliable "until a robust statistical relationship between given identified features and the learner's linguistic competence has been established" (p. 39). We believe our findings have made such a relationship clear.…”

Contemporary Second Language Assessment Edited by Dina Tsagari, Jayanti Banerjee. Hb $126.89, ISBN: 978‐0‐56763274‐6. 2016. London: Bloomsbury. vi + 318 pages

“…The findings showed that high-scoring essays contained more creative academic formulae, while low scoring essays included multiword units taken from the source material. Their findings support using small corpora to reveal the differences in test-takersp roficiency as measured by the use of multiword units (Park, 2014).…”

“…Their findings support using small corpora to reveal the differences in test-takersp roficiency as measured by the use of multiword units (Park, 2014). Adapting an interactional and sociolinguistic perspective, the author reports on validity evidence for a web-based test of pragmatics that includes multiturn discourse completion tasks as well as appropriateness judgments and single-turn items.…”

Empirical Learner Language and the Levels of the Common European Framework of Reference

“…However, recently there has been an increasing awareness of some crucial aspects at the interface of the fields, along with a growing readiness to integrate insights from all these areas into proficiency level–related research. This has led to much broader research perspectives (LTA and SLA: Alderson, ; Hulstijn et al., ; Leclercq, Edmonds, & Hilton, ; LTA and LCR: Barker, ; Barker et al., ; Callies et al., ; Callies & Götz, ; Callies & Paquot, ; Park, ; Taylor & Barker, ; LCR & SLA: Hasko & Meunier, ; Myles, ).…”