An investigation of item bias in the four-tier diagnostic test using Rasch model

Abstract

The existence of item bias in a set of measuring instruments can threaten the instrument’s validity. Based on the Rasch model, this study evaluated item bias in the four-tier heat and temperature diagnostic test (4T-HTDT). This study used a cross-sectional quantitative survey method. There were 241 students selected using a stratified random sampling technique. The 4T-HTDT instrument consisted of 20 items grouped into five concept groups. Students’ conceptual understanding was grouped into five categories, namely scientific knowledge (Rating=5), false positive (Rating=4), false negative (Rating=3), misconceptions (Rating=2), and lack of knowledge (Rating=1). The differential item functioning (DIF) score was used to evaluate item bias in the 4T-HTDT. Bias was reviewed based on the respondent’s gender, class, and school. The item has DIF if the probability value is <5%. The results showed that 35% (7 out of 20 items) spread over five groups of heat and temperature concepts were biased. However, excluding seven DIF items from the measurement set would not significantly affect the composition and distribution of items. Thus, the 13 items in the 4T-HTDT instrument are free from bias and can be used to evaluate the conceptual understanding of high school students.

Developing and validating the construct maps to assess mathematical proficiencies

“…The findings discovered that the multi-dimensional model had a significantly better statistical fit than the unidimensional model through the likelihood ratio Chi-squared G 2 (𝜒 2 =21.56, df=2) [23] as well as the Akaike information criterion (AIC) [24], and Bayesian information criterion (BIC) [25] had a lower value in multidimensional constructs for assessing MP, as shown in Table 3. The research indicates that it would be appropriate to diagnose mathematical proficiency in two dimensions in the real context [26]- [28]. Additionally, the results of the correlation matrix of MAP and SLO dimensions showed that there was a correlation between the two dimensions at 0.55.…”

What do my students need? Deep analysis of student's individual cognitive ability using person diagnostic map

“…The instrument used in this research was 20 multiple-choice items on heat and temperature material developed by Sukarelawan et al (Jumadi et al, 2023). Twenty items are spread into four concept groups, namely: (1) temperature (6 items), (2) expansion (4 items), (3) the effect of heat on temperature changes and changes in form (4 items), and (4) heat and heat transfer (6 items).…”

“…Various methods of analyzing students' conceptual understanding have been widely used. For example, interview methods (Fuchs & Czarnocha, 2016;Jankvist & Niss, 2018), multiple-choice (Dulger, 2017;Kusairi et al, 2022), two-level multiple choice (Atchia et al, 2022;Onder-Celikkanli & Tan, 2022;Wang et al, 2022), three-level multiple choice (Prodjosantoso et al, 2019;Yeo et al, 2022), and fourlevel multiple choice (Astuti et al, 2023;Atmaca Aksoy & Erten, 2022;Jumadi et al, 2023;Taban & Kiray, 2022). The approach to analyzing conceptual understanding is oriented to classical test theory (CTT).…”

Improving high school students' research skills through light diffraction experiments using the problem-based learning model

“…Racking analysis is used to analyze what types of skills change [29], [32], [33]. Grouping students' skill levels refers to the Logit Value of Person (LVP) [34], and grouping the difficulty level of skill aspects refers to the Logit Value of Item (LVI) [35], as shown in Table 1 and Table 2.…”