“Cognitive Diagnosis and Q-Matrices in Language Assessment”: A Commentary

Alderson, J. Charles

doi:10.1080/15434300903426748

Cited by 13 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The challenges of diagnosing L2 reading comprehension have long been echoed (e.g., Alderson, 2005, 2010). Previously, there has been little discussion of diagnosis, “of what it is, who does it, how it is done, and with what results” (Alderson et al, 2015, p. 1).…”

Section: Literature Reviewmentioning

confidence: 99%

“…Owing to the limitations of the retrofitted approach, the need for the design of purposebuilt diagnostic language tests has been echoed in the relevant literature (Alderson, 2010;Gierl & Cui, 2008;Jang 2009aJang , 2009bSawaki et al, 2009). Moreover, the conditions of our particular context, an EFL higher education setting where English is the medium of instruction, necessitated generating a diagnostic test of L2 reading that would help diagnose the reading ability profiles of the participants.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Examining the L2 reading comprehension ability of adult ELLs: Developing a diagnostic test within the cognitive diagnostic assessment framework

Toprak

Çakır

2020

Language Testing

View full text Add to dashboard Cite

Cognitive diagnostic assessment (CDA) has been applied to language assessment in a number of studies in which a diagnostic classification model (DCM) was retrofitted to the results of a non-diagnostic assessment. However, the need to apply CDA through utilization of an inductive rather than a retrofitted approach has been a recurrent theme in these studies. Thus, this study aimed to develop a diagnostic L2 reading comprehension test in English to investigate adult examinees’ reading performances in an EFL (English as a Foreign Language) academic setting. The test was based on a cognitive model of L2 reading comprehension and was administered to a sample of 1058 examinees across Turkey. The results were analyzed using log-linear cognitive diagnosis modeling (LCDM), which is one of the general DCM families subsuming other core DCMs. The findings of the study indicated that obtaining fine-grained diagnostic information about examinees’ performances in a given domain would be possible by coupling an adequate understanding of the construct with a CDA framework.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

mentioning

confidence: 99%

Examining the L2 reading comprehension ability of adult ELLs: Developing a diagnostic test within the cognitive diagnostic assessment framework

Toprak

Çakır

2020

Language Testing

View full text Add to dashboard Cite

show abstract

“…However, many limitations still stand in the way of its widespread adoption in the classroom. First and foremost, the process of content analysis is very complex, and among the published studies, there are discrepancies in the number of subskills that are identified even within the same test (Alderson, 2010). Because the statistical operations required for data analysis are quite complex, this process is also likely to be absolutely opaque to classroom teachers (Davidson, 2010).…”

Section: Theoretical Backgroundmentioning

confidence: 99%

Untitled

2020

The Journal of AsiaTEFL

View full text Add to dashboard Cite

show abstract

“…Hambleton et al treat all criterion-referenced tests as objectives-referenced, and argue that norm-referenced tests may be given criterion-referenced meaning if specific test items can be matched to individual objectives. This has generated a significant amount of psychometric research that aims to retrofit norm-referenced language tests with criterionreferenced meaning, from rule-space methodology (Tatsuoka and Buck, 1998) to fusion theory (Jang, 2009), none of which has been successful when dealing with complex constructs such as language competence (Alderson, 2010). Providing substantive meaning to test scores requires validation by design, rather than validation by retrofit (Fulcher, 2013a).…”

Section: The Corruption Criterionmentioning

confidence: 99%

Limited aspects of reality: Frames of reference in language assessment

Fulcher¹,

Svalberg²

2013

ijes

View full text Add to dashboard Cite

Language testers operate within two frames of reference: norm-referenced (NRT) and criterion-referenced testing (CRT). The former underpins the world of large-scale standardized testing that prioritizes variability and comparison. The latter supports substantive score meaning in formative and domain specific assessment. Some claim that the criterion-referenced enterprise is dead, save its legacy in score reporting (Davidson, 2012, p. 198). We argue that announcing the demise of CRT is premature. But we do acknowledge that what now passes as CRT is in fact not criterion-referenced, but is based upon a corruption of the original meaning of "criterion" as domain-specific performance. This distortion took place when NRT co-opted the term "standard" to serve as a rationale for the measurement enterprise of establishing cut-scores to retrofit NR tests with meaning derived from external scales. The true heirs of the CRT movement are researchers who base test design in the careful analysis of construct and content in domain specific communication.

show abstract

“Cognitive Diagnosis and Q-Matrices in Language Assessment”: A Commentary

Cited by 13 publications

References 11 publications

Examining the L2 reading comprehension ability of adult ELLs: Developing a diagnostic test within the cognitive diagnostic assessment framework

Examining the L2 reading comprehension ability of adult ELLs: Developing a diagnostic test within the cognitive diagnostic assessment framework

Untitled

Limited aspects of reality: Frames of reference in language assessment

Contact Info

Product

Resources

About