The Development of an Instrument for Longitudinal Learning Diagnosis of Rational Number Operations Based on Parallel Tests

Educational Measurement

2021

Self Cite

In learning diagnostic assessments, the attribute hierarchy specifies a sequential network of interrelated attribute mastery processes, which makes a test blueprint consistent with the cognitive theory. One of the most important functions of attribute hierarchy is to guide or limit the developmental direction of students and then form a hierarchical learning trajectory. To address the issue that the existing longitudinal learning diagnosis models cannot track the development of hierarchical attributes, this study proposes a new hierarchical longitudinal learning diagnostic modeling approach and two sample models. Compared to the longitudinal learning diagnosis models that do not consider the attribute hierarchy, the proposed models, by taking the sequential mastery tree into account, can accommodate various attribute hierarchies and simultaneously track an individual's learning developmental trajectory. An empirical study was conducted to illustrate the advantages of the proposed model. The results mainly indicated that the proposed model can properly diagnose the development of hierarchical attributes in longitudinal assessments. A simulation study was further conducted to explore the model parameter recovery of the proposed models.

Section: Resultsmentioning

confidence: 99%

Section: An Empirical Studymentioning

confidence: 99%

A Longitudinal Diagnostic Model with Hierarchical Learning Trajectories

Educational Measurement

2021

Self Cite

“…Since the development process is not the focus of this study, we only give a brief introduction to it, and more details about it can be found in Tang and Zhan (2020). The development process contains three main phases: (a) the Q-matrix (Tatsuoka, 1983) construction and item development, (b) the pilot test for item quality monitoring (the preliminary screening of the items is conducted mainly according to item difficulty and discrimination based on classical test theory), and (c) the formal test for test quality control (including the Q-matrix validation [de la Torre, 2008], reliability and validity testing [W. Wang et al, 2015], differential item functioning checking [Hou et al, 2014], and parallel tests checking).…”

Section: Instrumentmentioning

confidence: 99%

Does Diagnostic Feedback Promote Learning? Evidence From a Longitudinal Cognitive Diagnostic Assessment

Tang

2021

AERA Open

Self Cite

Assessment for learning emphasizes the importance of feedback to promote learning. To explore whether cognitive diagnostic feedback (CDF) promotes learning and whether it is more effective than traditional feedback in promoting learning, this study conducted a quasi-experiment by utilizing a longitudinal cognitive diagnostic assessment to compare the effect of three feedback modes on promoting learning, including CDF, correct–incorrect response feedback (CIRF), and no feedback. The results provided some evidence for the conclusion that CDF can promote students’ learning and is more effective than CIRF in promoting learning, especially in more challenging areas of knowledge.

“…Three parallel tests had the same Q‐matrix (see Figure 4), which contained 18 dichotomous items (i.e., 54 items in total) and six fine‐grained attributes, namely, (A1), rational number; (A2), related concepts of the rational number; (A3), number axis; (A4), addition and subtraction of rational numbers; (A5), multiplication and division of rational numbers; and (A6), mixed operation of rational numbers. More details about this instrument used in the quasi‐experiment can be found in Tang and Zhan (2020). The results of this quasi‐experiment mainly indicated that both learning diagnostic feedback and traditional correct‐incorrect response feedback could promote student learning, but the former was significantly more effective than the latter.…”

Section: An Empirical Examplementioning

confidence: 99%

Refined Learning Tracking with a Longitudinal Probabilistic Diagnostic Model

Educational Measurement

2020

Self Cite

Refined tracking allows students and teachers to more accurately understand students’ learning growth. To provide refined learning tracking with longitudinal diagnostic assessment, this article proposed a new model by incorporating probabilistic logic into longitudinal diagnostic modeling. Specifically, probabilistic attributes were used instead of binary attributes to model the latent variables that affect students’ performance. Thus, in the proposed model, attribute‐level growth can be quantified in a more refined manner. The feasibility of the proposed model was examined using simulated data. The results mainly indicated that the model parameters for the proposed model could be well recovered. An empirical example was conducted to illustrate the applicability and advantages of the proposed model. The results mainly indicated that when distinguishing the level of students, the diagnostic results of the proposed model and the conventional longitudinal diagnostic model for binary attributes displayed a high degree of consistency; however, the former could provide more refined description of growth and a better model‐data fit than the latter.