Analyzing the science achievement test: Perspective of classical test theory and Rasch analysis

Abstract

Assessment in teaching and learning allows both instructor and students to evaluate the achievement, performance and improve upon it. The achievement test appears as one of the most commonly used assessment tools to determine students' gains in the cognitive domain. The study analysed science achievement test (SAT), a multiple-choice test for grade 10 senior secondary school students. The 50 SAT items were developed in compliance with the Malaysian science curriculum specification. The face and content validity of SAT were validated by experts in the science field. The SAT was administered with 50 students in a pilot study. The collected data was analysed using item analysis based on classical test theory (CTT) and Rasch analysis. In terms of difficulty index (p), item analysis showed that there were five difficult items, 33 moderately difficult items, and 12 easy items. Findings also reported eight items presented as poor items due to the poor discriminating power. The SAT showed a good coefficient of 0.862 for Kuder-Richardson 20 (KR-20) and 0.851 for the split half method. While the Rasch analysis showed a reliable and good separation for both item and person. Besides, the Rasch analysis displayed that there are two items should be refreshed or removed. The study revealed that the SAT was a solid, valid, and reliable tool, which is suitable to be used to measure the student’s achievement in science.

Exploring the Accuracy and Consistency of a School Readiness Assessment Tool for Preschoolers: Reliability, Validity and Measurement Invariance Analysis

“…CTT is the oldest measurement theory that assumes a person's test score is the sum of their true score (actual ability) and measurement error (Siregar and Panjaitan 2022). It analyzes the reliability, validity, and sources of measurement error, with the true score representing the individual's actual ability and the measurement error reflecting the variability in observed scores unrelated to the true score (Haw et al 2022). However, CTT does not account for item difficulty or variability in individual differences in ability levels (Ayanwale et al 2022) and MI testing (Siregar and Panjaitan 2022).…”

Diagnostic Assessment of School Readiness

“…CTT is the oldest measurement theory that assumes a person's test score is the sum of their true score (actual ability) and measurement error (Siregar & Panjaitan, 2022). It analyzes the reliability, validity, and sources of measurement error, with the true score representing the individual's actual ability and the measurement error reflecting the variability in observed scores unrelated to the true score (Haw et al, 2022). However, CTT does not account for item difficulty or variability in individual differences in ability levels (Ayanwale et al, 2022) and MI testing (Siregar & Panjaitan, 2022).…”

Which one is more accurate, BILOG or R program? (a comparison for score test equating)

“…This can cause students with lower abilities to get higher scores when working on easy questions, while students with higher abilities get lower scores when taking more difficult tests. This situation can cause measurement errors when examinee's ability and the item's features could predict or explain the success of an examinee [10]. If there is a measurement error, the evaluation results do not show the actual level of student competence.…”