Comparative Analysis of Classical Test Theory and Item Response Theory Based Item Parameter Estimates of Senior School Certificate Mathematics Examination

Abstract

The study compared Classical Test Theory (CTT) and Item Response Theory (IRT)-estimated item difficulty and item discrimination indices in relation to the ability of examinees in Senior School Certificate Examination (SSCE) in Mathematics with a view to providing empirical basis for informed decisions on the appropriateness of statistical and psychometric tests. The study adopted ex-post-facto design. A sample of 6,000 students was selected from the population of 35,262 students who sat for the NECO SSCE Mathematics Paper 1 in 2008 in Osun State, Nigeria. An instrument consisting of 60-multiple-choice items, May/June 2008 NECO SSCE Mathematics Paper 1 was used. Three sampling plans: random, gender and ability sampling plans were employed to study the behaviours of the examinees scores under the CTT and IRT measurement frameworks. BILOG-MG 3 was used to estimate the indices of item parameters and SPSS 20 was used to compare CTT- and IRT-based item parameters. The results showed that CTT-based item difficulty estimates and oneparameter IRT item difficulty estimates were comparable (the correlations were generally in the -0.702 to -0.988 range in large sample and -0.622 to - 0.989 range in small sample). Results also indicated that CTT-based and two-parameter IRT-based item discrimination estimates were comparable (the correlations were in the 0.430 to 0.880 ranges in large sample and 0.531 to 0.950 range in small sample). The study concluded that CTT and IRT were comparable in estimating item characteristics of statistical and psychometric tests and thus could be used as complementary procedures in the development of national examinations

Development of Vector Representation Test Instrument for Senior High School Students in Realizing Learning Outcomes

“…The same thing is shown by Ayanwale (2018) that IRT approaches with PCM 1-PL in item analyze more better than (Classical Test Theory) CTT. Problems with classical test theory will have an impact on the level of ability of students that is difficult to know (Awopeju & Afolabi, 2016).…”

“…The disadvantages of classical measurement theory include the characteristics of test items such as the level of difficulty and the power of differences that depend on students (Persichitte, 2016). Problems with classical measurement theory will have an impact on the level of ability of students that is difficult to know (Awopeju & Afolabi, 2016). Problems that arise in classical measurement theory can be solved by applying modern measurement theory, namely the item response theory approach (Item Response Theory, IRT) (Baker, 2001).…”

Competitive Learning Using a Three-Parameter Logistic Model

“…Among the extensive literature that considers the application of tests of abilities and knowledge in university settings, the so-called Item Response Theory (IRT) can be found. This theory, among other things, serves to analyze curricular content, adequate item design, the recognition of teaching-learning problems and the identification of students with low or high academic ability (Awopeju, Afolabi, 2016;Balmori et al, 2011;DiBattista, Kurzawa, 2011;Ingale et al, 2017;Mitra et al, 2009;Rao et al, 2016;Romero et al, 2015).…”

Analysis of Competitive Learning at University Level in Mexico via Item Response Theory

“…Various studies have analyzed knowledge examinations at a university level using Item Response Theory (IRT) to evaluate course topics, analyzing both the quality of the items and difficulties with learning, as well as identifying badly performing students (Awopeju & Afolabi, 2016;Balmori, Delgadillo, & Méndez, 2011;DiBattista & Kurzawa, 2011;Ingale, Giri, & Doibale, 2017;Mitra, Nagaraja, Ponnudurai, & Judson, 2009;Rao, Kishan Prasad, Sajitha, Permi, & Shetty, 2016;Romero, Rojas, Domínguez, Pérez, & Sapsin, 2015).…”

“…Of the studies on partial knowledge tests or those graduating from undergraduate degree programs, various have used IRT (Awopeju & Afolabi, 2016;Balmori et al, 2011;DiBattista & Kurzawa, 2011;Escudero, Reyna, & Morales, 2000;Gajjar, Sharma, Kumar, & Rana, 2014;Ingale et al, 2017;Marie & Edannur, 2015;Mitra et al, 2009;Rao et al, 2016;Romero et al, 2015). However, from the perspective of the authors of this study, this is the first paper to analyze a test taken from a statistics knowledge competition using IRT, by means of which it sought to identify university students' competitive learning.…”