What does the analysis of C-test gaps tell us about the construct of a C-test? A comparison of foreign and heritage language learners’ performance

Abstract

In view of the ubiquitous increase in the use of C-tests, which are almost unanimously believed to measure general language proficiency, this study investigates whether the aspects of language proficiency tapped into by the C-test format are the same when the test is taken by a learner population other than that of foreign language learners. Specifically, we conducted a differential functioning analysis and compared the types of mistakes that 113 foreign language learners of Russian made when completing C-test gaps, with the performance of 89 heritage language learners on the same C-test. The results showed that almost half of the C-test gaps are biased towards either learner group. In addition, the error analysis for a number of the biased items demonstrated that, although heritage language learners seem to have an advantage in reconstructing the meaning of C-test gaps, they fail to translate their recognition skills into producing the right form. Furthermore, the study reveals a possible sensitivity of the C-test construct to the traditionally used dichotomous scoring method. We conclude with a discussion that includes the implications of the results regarding the construct measured by the C-test and the possible consequences for its actual use.

Does bilingualism correlate with or predict higher proficiency in L3 English? A contrastive study of monolingual and bilingual learners

“…Therefore, the C-test scale ranged from 0 to 90. C-test measures are not uncontroversial, as Drackert and Timukova (2020) have demonstrated in a recent study. They used a Russian C-test with two different learner populations, namely foreign language learners of Russian and Russian heritage speakers.…”

What modulates the acquisition of difficult structures in a heritage language? A study on Portuguese in contact with French, German and Italian

“…Therefore, cloze-tests are viewed as integrative assessment tools, since participants have to access large parts of their linguistic knowledge to reconstruct the missing gap (Chung & Ahn, 2019). Participants need to “have the item in their vocabulary, to identify the item correctly based on the context, and to produce its correct grammatical and orthographical form” (Drackert & Timukova, 2020, p. 123). Cloze-tests and their sub-types (such as c-tests) have been extensively used across different educational contexts such as language placement tests and as measures of language proficiency in experimental linguistic research (see Grotjahn & Drackert, 2020 for an updated c-test bibliography; see also Brown, 1980; Tremblay, 2011; Tremblay & Garrison, 2008).…”

“…However, most research on cloze-tests is conducted with adult foreign language learners, who acquired the assessed language in formal instruction settings. The validation of cloze-tests for other types of speakers, such as HL speakers, has been initiated only recently (Drackert & Timukova, 2020; Luchkina, Ionin, Lysenko, Stoops & Suvorkina, 2021). Drackert and Timukova (2020) show that late L2 learners perform better than HL speakers in a cloze-test, mainly because the latter exhibit some deficits in orthographical knowledge (see also Mehlhorn, 2016).…”

Elicited Imitation as a Measure of L2 Proficiency

“…A particular attraction of EITs is their practicality: They can be administered in about 10 minutes per participant, whereas standardized proficiency tests can take an hour or longer. In fact, EITs are even shorter than other short-cut measures of L2 proficiency, such as cloze tests (Tremblay, 2011) or C-tests (Drackert & Timukova, 2020). Ortega et al (2002) originally developed parallel EITs for English as well as German, Japanese, and Spanish.…”