What CDM Can Tell About What Students Have Learned: An Analysis of TIMSS Eighth Grade Mathematics

Abstract

International trended assessments have long attempted to provide instructional information to educational researchers and classroom teachers. Studies have shown that traditional methods of item analysis have not provided specific information that can be directly applicable to improve student performance. To this end, cognitive diagnosis models (CDMs) were developed to identify the presence or the absence of multiple finegrained skills or attributes. This study reanalyzed the TIMSS 2003 8 th grade mathematics assessment by applying a cognitive diagnosis modeling approach to provide useful information about test takers. Specifically, the deterministic, inputs, noisy, "and" gate (DINA; Junker & Sijtsma, 2001) model was used to provide more valuable diagnostic information about how well students perform on underlying skills and cognitive processes necessary to answer mathematics problems. Results showed that the interpretable benefits of using the DINA discrimination index provided additional information about comparative differences between the U.S. and Korea.

International Comparative Study on PISA Mathematics Achievement Test Based on Cognitive Diagnostic Models

“…Combining modern statistical methods with cognitive theories, CDMs have been widely utilized in educational and psychological assessment. One of the advantages of using CDMs is their ability to identify the strengths and weaknesses in a set of fine-grained skills (or attributes) when difficulty exists in inferring skill mastery profiles of examinees through traditional methods, such as classical test theory (CTT) and IRT (Choi et al, 2015). Therefore, CDMs have been developed to provide fine-grained information for researchers and educators on the cognitive skills or attributes that are required to solve a particular item, allow applications in various instructional practices, and resolve the limitations that exist in the IRT and CTT models (De La Torre, 2009).…”

Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment

“…While these findings are all based on the TIMSS 2007 mathematics assessment, we believe that they might have implications beyond it. For example, in educational diagnostic assessment, quite a few studies (e.g., [ 31 , 32 , 42 ]) have applied the DINA model. For example, Sun, Suzuki, and Kakinuma [ 42 ] analyzed the fraction test for sixth grade students using the DINA model, and then gave them actual feedback based on the DINA model diagnosis.…”

“…They found that the DINA model fit better than did the IRT models in terms of both AIC and BIC. Second, Choi, Lee, and Park [ 32 ] compared Korean and U.S. national samples using the DINA model for the TIMSS 2003 eighth grade mathematics assessment. They did not conduct a formal likelihood-based model comparison, and instead examined the differences in attribute mastering profile between the two countries.…”

Investigating Preservice Middle School Mathematics Teachers’ Competencies in Statistics and Probability in Terms of Various Variables

“…Bilişsel tanı modellerinin güçlü olduğu yönlerden birisi, az sayıda test maddesi kullanılsa bile KTK'ya kıyasla daha güvenilir tahminler sunmalarıdır (Templin & Bradshaw, 2013). Son yıllarda araştırmacılar bilişsel tanı modellerini kullanarak öğrencilerin (örn., Choi, Lee & Park, 2015;Dogan & Tatsuoka, 2008;Im & Park, 2010;Lee, Park & Taylan, 2011;Sen & Arican, 2015), öğretmenlerin (örn., Bradshaw, Izsak, Templin & Jacobson, 2014) ve öğretmen adaylarının (örn., Arican & Kuzu, 2019) çeşitli matematik konularından elde ettikleri sonuçlar üzerine tanılayıcı değerlendirmede bulunmuşlardır.…”