The cagey C-test construct: Some evidence from a meta-analysis of correlation coefficients

McKay, Todd; Teimouri, Yasser; Sağdıç, Ayşenur; Salen, Bradford; Reagan, Derek; Malone, Margaret E.

doi:10.1016/j.system.2021.102526

Cited by 6 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li and Zhang (2021), for example, recently meta-analyzed the relationship between self-assessed proficiency and measures of language performance, observing an overall correlation of r = .47. McKay et al (2021), however, observed much stronger evidence of the criterion-related validity in his meta-analysis of the relationship between C-tests and other measures of proficiency ( r corrected = .87).…”

Section: Discussionmentioning

confidence: 96%

Elicited Imitation Tasks as a Measure of L2 Proficiency

Kostromitina

Plonsky

2021

Stud Second Lang Acquis

View full text Add to dashboard Cite

Elicited imitation tasks (EITs) have been proposed and examined as a practical measure of second language (L2) proficiency. This study aimed to provide an updated and comprehensive view of the relationship between EITs and other proficiency measures. Toward that end, 46 reports were retrieved contributing 60 independent effect sizes (Pearson’s r) that were weighted and averaged. Several EIT features were also examined as potential moderators. The results portray EIT as a generally consistent measure of L2 proficiency (r = .66). Among other moderators, EIT stimuli length was positively associated with stronger correlations. Overall, the findings provide support for the use of EITs as a means to greater consistency and practicality in measuring L2 proficiency. In our Discussion section, we highlight the need for more transparent reporting and provide empirically grounded recommendations for EIT design and for further research into EIT development.

show abstract

Section: Discussionmentioning

confidence: 96%

Elicited Imitation Tasks as a Measure of L2 Proficiency

Kostromitina

Plonsky

2021

Stud Second Lang Acquis

View full text Add to dashboard Cite

show abstract

“…3 3 This concurrent calibration is based on the unidimensionality assumption that all three task types are measuring the same construct. Although it is difficult to argue that the C‐test items measure exactly the same construct as the LR and LC items do, there is theoretical (e.g., Babaii & Ansary, 2001) and empirical (e.g., McKay et al, 2021) evidence that establishes the effectiveness of the C‐test items as a measure of general language proficiency. We also examined the factor structure of both forms using the single‐factor confirmatory model and observed acceptable fit in both cases (RMSEA of 0.058 and 0.059 for Forms 1 and 2, respectively).…”

mentioning

confidence: 99%

The Impact of Using Synthetically Generated Listening Stimuli on Test‐Taker Performance: A Case Study With Multiple‐Choice, Single‐Selection Items

Choi

2022

ETS Research Report Series

View full text Add to dashboard Cite

Synthetically generated speech (SGS) has become an integral part of our oral communication in a wide variety of contexts. It can be generated instantly at a low cost and allows precise control over multiple aspects of output, all of which can be highly appealing to second language (L2) assessment developers who have traditionally relied upon human voice actors for recording audio materials. Nevertheless, SGS is not widely used in L2 assessments. One major concern in this use case lies in its potential impact on test-taker performance: Would the use of SGS (as opposed to using human voice actor recordings) change how test takers respond to an item? In this study, we investigated using SGS as stimuli for English L2 listening assessment items on test-taker performance. The data came from a pilot administration of multiple new task types and included 653 test takers' responses to two versions of the same 13 items, differing only in terms of their listening stimuli: a version using human voice actor recordings and the other version with SGS files. Multifaceted comparisons between test takers' responses across the two versions showed that the two versions elicited remarkably comparable performance. The comparability provides strong empirical evidence for the use of SGS as a viable alternative for human voice actor recordings in the immediate domain of L2 assessment as well as related domains such as learning material and research instrument development.

show abstract