Optimal classification methods for diagnosing latent skills and misconceptions for option-scored multiple-choice item quizzes

Stout, William; Henson, Robert A.; DiBello, Lou

doi:10.1007/s41237-022-00172-0

Cited by 5 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This can be achieved by adopting appropriate cognitive diagnostic models (CDMs), which are latent class models used for classifying students based on their skill profiles. Current CDA studies in L2 listening focus on identifying whether a particular learner has mastered some specific language subskills (known as attributes) such as “listening for details” or “listening for main ideas” based on the correct responses or options (in the case of multiple choice questions, MCQ hereafter) (e.g., Min and Xiong, 2019 ; He et al, 2022 ), but they undervalue the rich information of students’ incorrect options or distractors which are often intentionally designed and likely to be chosen to reveal specific problems in knowledge or skills ( Ozaki et al, 2019 ), and can also provide useful diagnostic feedback ( Stout et al, 2022 ). Therefore, both incorrect and correct listening responses (or MCQ distractors) deserve equal attention and treatment.…”

Section: Introductionmentioning

confidence: 99%

Cognitive diagnostic assessment of EFL learners’ listening barriers through incorrect responses

Meng,

Wang,

Zhao

2023

Front. Psychol.

View full text Add to dashboard Cite

English as Foreign Language (EFL) learners’ cognitive processes have been a research focus in listening assessment. Most studies use correct responses as data, but undervalue the rich information of the incorrect answers or options (in the case of multiple choice questions, MCQ). However, the MCQ distractors are often intentionally designed to reveal learners’ problems or barriers. In order to diagnose the EFL learners’ listening barriers through incorrect responses, Cognitive Diagnostic Models (CDMs) for bugs were adopted, hence the name Bug-CDMs. First, five EFL listening barrier attributes were identified and two Bug Q-matrices were developed to comparatively analyze the learner’s responses with different Bug-CDMs. The results revealed that Bug-GDINA was the optimal model, and the most prevalent barriers were semantic understanding and vocabulary recognition. These barriers confirmed both compensatory and non-compensatory relationships in causing listening comprehension failures. The study proved the feasibility of Bug-GDINA in diagnosing listening barriers from the incorrect responses. Limitations and suggestions for further research were also proposed.

show abstract

Section: Introductionmentioning

confidence: 99%