Duolingo English Test :Technical Manual

Cardwell, Ramsey; Naismith, Ben; LaFlair, Geoffrey T.; Nydick, S. E.

doi:10.46999/cqng4625

Cited by 14 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We analyzed responses from a sample of the Duolingo English Test test takers, a computerized-adaptive assessment of English language proficiency designed to support high-stakes decisions in English-medium settings (Cardwell et al 2023). The sample consists of 295,157 test sessions from 222,568 unique test takers (some repeat the assessment).…”

Section: Datamentioning

confidence: 99%

“…The dataset also includes each test taker's score on the speaking and writing portions of the assessment, which we use in some models as predictors of dictation proficiency via latent regression. Details on the tasks contributing to these speaking and writing scores and other technical details about the assessment can be found in Cardwell et al (2023).…”

Section: Datamentioning

confidence: 99%

“…For instance, when a continuous feature has a bounded distribution (e.g., between 0 and 1), it may have a nontrivial probability of being 0 and/or 1 when computed from a sample of responses from some target population. An example that we consider in this study arises from the use of edit distance (Levenshtein 1965) to evaluate the accuracy of a response to a dictation task, in which a test taker is asked to listen to a target sentence in English and then type the sentence in English (Buck 2001;Cardwell et al 2023). If the typed response matches the target sentence, one could assign a grade of 1.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses

Zopluoglu,

Lockwood

2024

J. Intell.

View full text Add to dashboard Cite

Language proficiency assessments are pivotal in educational and professional decision-making. With the integration of AI-driven technologies, these assessments can more frequently use item types, such as dictation tasks, producing response features with a mixture of discrete and continuous distributions. This study evaluates novel measurement models tailored to these unique response features. Specifically, we evaluated the performance of the zero-and-one-inflated extensions of the Beta, Simplex, and Samejima’s Continuous item response models and incorporated collateral information into the estimation using latent regression. Our findings highlight that while all models provided highly correlated results regarding item and person parameters, the Beta item response model showcased superior out-of-sample predictive accuracy. However, a significant challenge was the absence of established benchmarks for evaluating model and item fit for these novel item response models. There is a need for further research to establish benchmarks for evaluating the fit of these innovative models to ensure their reliability and validity in real-world applications.

show abstract

Section: Datamentioning

confidence: 99%

Section: Datamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses

Zopluoglu,

Lockwood

2024

J. Intell.

View full text Add to dashboard Cite

show abstract

“…Generally speaking, the test places emphasis on natural and fluent language use, effective expression, and the simultaneous use of multiple language skills. Its goal is to provide a comprehensive assessment of language proficiency that is reflective of real-world language use (Cardwell et al, 2022).…”

Section: Consequential Validitymentioning

confidence: 99%

Predictability of Duolingo English mock test for Chinese college-level EFLs: using assessment use argument

Ma,

Zhang

2024

Front. Educ.

View full text Add to dashboard Cite

Online international language tests have been gaining popularity since the COVID-19 pandemic. This study explored the predictive ability of the mock test of Duolingo English Test (DET) for Chinese students. College-level EFLs (N = 42) participated in this study and completed the official online mock test. Through quantitative analysis, the findings demonstrated that mock scores were significantly correlated with TOEFL or IELTS results. However, given that the mock test scores are significantly lower than expected, the consequential validity needs further analysis. Using Assessment Use Argument in qualitative analysis, the study established the claim, warrants, evidence, and potential rebuttals regarding the consequential validity of Duolingo English mock test. With further analysis on test specification, the characterization of Target Language Use domain of DET reading test items were investigated. The results indicate that although the DET mock test is relatively more accessible, it needs modification to become more suitable for Chinese EFLs. Suggestions and limitations on Chinese students using DET and its mock test were also discussed.

show abstract

AI in the Assessment Ecosystem: A Human–Centered AI Perspective

von Davier,

Burstein

2024

Intelligent Systems Reference Library

View full text Add to dashboard Cite

Duolingo English Test :Technical Manual

Cited by 14 publications

References 31 publications

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses

Predictability of Duolingo English mock test for Chinese college-level EFLs: using assessment use argument

AI in the Assessment Ecosystem: A Human–Centered AI Perspective

Contact Info

Product

Resources

About