Does Diagnostic Feedback Promote Learning? Evidence From a Longitudinal Cognitive Diagnostic Assessment

Tang, Fang; Zhan, Peida

doi:10.1177/23328584211060804

Cited by 19 publications

(24 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The empirical research findings showed promise that teachers and students can use feedback from these models to decide next steps in learning in different learning domains, such as reading (e.g., Kim, 2015 ), listening (e.g., Min and He, 2022 ), writing (e.g., Xie, 2017 ), and mathematics (e.g., Wu, 2019 ). Tang and Zhan (2021) also showed that CDM promotes students’ learning. However, CDMs have not been applied to the energy domain to characterize or support students’ learning.…”

Section: Introductionmentioning

confidence: 90%

Measuring students’ learning progressions in energy using cognitive diagnostic models

Zhou

Traynor

2022

Front. Psychol.

View full text Add to dashboard Cite

This study applied cognitive diagnostic models to assess students’ learning progressions in energy. A Q-matrix (i.e., an item attribute alignment table) was proposed based on existing literature about learning progressions of energy in the physical science domain and the Trends in International Mathematics and Science Study (TIMSS) assessment framework. The Q-matrix was validated by expert review and real data analysis. Then, the deterministic inputs, noisy ‘and’ gate (DINA) model with hierarchical relations was applied to data from three jurisdictions that had stable, defined science curricula (i.e., Australia, Hong Kong, and Ontario). The results suggested that the hypothesized learning progression was consistent with the observed progression in understanding the energy concept. We also found similarities in students’ attribute mastery across the three jurisdictions. In addition, we examined the instructional sensitivity of the selected item. We discuss several curriculum-related issues and student misconceptions that may affect students’ learning progressions and mastery patterns in different regions of the world.

show abstract

Section: Introductionmentioning

confidence: 90%

Measuring students’ learning progressions in energy using cognitive diagnostic models

Zhou

Traynor

2022

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Moreover, traditional assessments do not have the contextual affordance of reflecting on HOTS levels for skill diagnosis. On the contrary, detailed cognitive modeling in comprehension is parallel to cognitive diagnostic assessment that can provide each learner's developmental trajectory in reading (Tang and Zhan, 2021). It provides ample systematic information to the teacher and learner about how and where to focus on improvement.…”

Section: Hots-based Reading Comprehensionmentioning

confidence: 99%

Modeling higher order thinking skills and metacognitive awareness in English reading comprehension among university learners

Aziz

Rawian²

2022

Front. Educ.

View full text Add to dashboard Cite

This study aimed to examine the levels of Pakistani university learners' higher order thinking skills (HOTS) in English reading comprehension and to explore how explicit assessments of HOTS can reveal learners' strengths and weaknesses in reading as compared to the traditional assessment of reading. Secondly, the study analyzed the effect of learners' metacognitive awareness on reading on the relationship between their HOTS levels and reading performance. It used a mixed-method approach. In the quantitative phase, an explicit HOTS-based reading test was designed using Bloom's taxonomy. Using random stratified sampling, it was administered to the learners of Bachelor of Science in English (n = 200). Stratified sampling was performed for investigating the gender gap in HOTS levels. The learners' metacognitive awareness of reading was analyzed using Survey of Reading Strategies. Descriptive analysis, independent sample t-test, correlation analysis, and moderator analysis were conducted. It was found that the learners' HOTS levels correlated with their reading performance where creative skill was the strongest, and evaluating skill was the weakest. Metacognitive awareness of reading positively moderated the relationship between the learners' HOTS levels and reading performance. In the qualitative phase, purposive stratified sampling was applied. A subsample (n = 6) was extracted from the core sample based on the high, average, and low reading performance, and three groups were formed. A semi-structured interview was conducted to know learners' opinions on using HOTS-based assessment in reading and their cognitive and metacognitive abilities. The thematic analysis of the qualitative data revealed that three out of six learners did not approve of the way they were assessed by the university. They informed that they had issues with cognitive and metacognitive control of English texts. The findings revealed that the majority of learners suffered low HOTS levels in English reading. The study offers recommendations for English as a second language (ESL) teachers, assessors, and policymakers to improve learner performance to the possible optimal level, especially in underdeveloped contexts that use traditional reading assessments.

show abstract

“…Because they incorporate additional variables beyond scored item responses, CDMs have the potential to provide richer diagnostic feedback than traditional item response theory (IRT) models (Ma & de la Torre, 2020). Such diagnostic feedback has been examined for its ability to promote students’ learning outcomes (Tang & Zhan, 2021). For example, teachers can use diagnostic feedback to design and improve remedial instruction, with the goal of efficient and targeted teaching interventions for an individual or group of students (e.g., Wang et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…In other words, the diagnostic results of most existing CDMs cannot fully reflect individual differences in problem-solving behaviors, which may further limit the amount of information contained in the existing diagnostic feedback. A number of studies in the field of learning science have shown that the richer the information contained in the feedback, the more conducive it is to promote student learning (e.g., Dunn et al, 2012; Maddox et al, 2008; Tang & Zhan, 2021).…”

Section: Introductionmentioning

confidence: 99%

Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis

Zhan

Man²,

Wind

et al. 2022

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

Respondents’ problem-solving behaviors comprise behaviors that represent complicated cognitive processes that are frequently systematically tied to one another. Biometric data, such as visual fixation counts (FCs), which are an important eye-tracking indicator, can be combined with other types of variables that reflect different aspects of problem-solving behavior to quantify variability in problem-solving behavior. To provide comprehensive feedback and accurate diagnosis when using such multimodal data, the present study proposes a multimodal joint cognitive diagnosis model that accounts for latent attributes, latent ability, processing speed, and visual engagement by simultaneously modeling response accuracy (RA), response times, and FCs. We used two simulation studies to test the feasibility of the proposed model. Findings mainly suggest that the parameters of the proposed model can be well recovered and that modeling FCs, in addition to RA and response times, could increase the comprehensiveness of feedback on problem-solving-related cognitive characteristics as well as the accuracy of knowledge structure diagnosis. An empirical example is used to demonstrate the applicability and benefits of the proposed model. We discuss the implications of our findings as they relate to research and practice.

show abstract

Does Diagnostic Feedback Promote Learning? Evidence From a Longitudinal Cognitive Diagnostic Assessment

Cited by 19 publications

References 56 publications

Measuring students’ learning progressions in energy using cognitive diagnostic models

Measuring students’ learning progressions in energy using cognitive diagnostic models

Modeling higher order thinking skills and metacognitive awareness in English reading comprehension among university learners

Cognitive Diagnosis Modeling Incorporating Response Times and Fixation Counts: Providing Comprehensive Feedback and Accurate Diagnosis

Contact Info

Product

Resources

About