Investigating Item Bias in a CS1 Exam with Differential Item Functioning

Davidson, Matthew; Wortzman, Brett; Ko, Amy J.; Li, Min

doi:10.1145/3408877.3432397

Cited by 5 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Not only are different assessments required because of the course material, but to promote equity among student populations. Differential Item Functioning methods have not been used with either the FCS1 or SCS1, but should be to understand how different student groups perform on the assessments [11]. Constructing courses and curricula to promote equity will only work if the assessments for measuring learning and achievement have also been updated to support equity goals.…”

Section: Discussion: Community Needs and The Future Of Assessment In Computing Educationmentioning

confidence: 99%

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

Parker

Guzdial

Tew³

2021

Proceedings of the 17th ACM Conference on International Computing Education Research

View full text Add to dashboard Cite

Validated instruments of knowledge are important for creating an accepted and shared measurement for the most important part of education -student learning. The Force Concept of Inventory from Physics Education Research has had a significant impact across STEM education. The Foundational CS1 (FCS1) and Second CS1 (SCS1) assessments were created and validated to further computing education research. Now, ten years after the publication of the FCS1 assessment and five years after the release of the SCS1 assessment, we can trace the use and the impact that these validated instruments have had on the needs and knowledge of the computing education community. In this paper, we examine how the FCS1 and SCS1 assessments have been used and adapted. We use this discussion to guide a comparison between our field and physics education to give a sense of direction to future research. In looking back on the use of these validated instruments, we can better understand our needs in computing education research and about our future needs in assessment. CCS CONCEPTS• Social and professional topics → Student assessment.

show abstract

Section: Discussion: Community Needs and The Future Of Assessment In Computing Educationmentioning

confidence: 99%

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

Parker

Guzdial

Tew³

2021

Proceedings of the 17th ACM Conference on International Computing Education Research

View full text Add to dashboard Cite

show abstract

“…The framework of DIF has become an integral component of test validation methodology and the study of test fairness. All DIF methods work by comparing how two groups perform on each item in an exam, after matching examinees on a criterion (Davidson, Ko, Wortzman & Li, 2021). That criterion can be the total score on ability, or some other variable.…”

Section: Differential Item Functioningmentioning

confidence: 99%

Differential Item Functioning Detection Methods: An Overview

Ohiri,

Momoh,

Christopher

et al. 2024

gengpi

View full text Add to dashboard Cite

With the increasing concerns over the fairness of educational and psychological test items, differential item functioning (DIF) approach has been widely used to find out items that are biased. In the educational and psychological measurement settings, a test item is recognized as differentially functioning across groups when the probability of an examinee's response to it depends on group membership. This paper explored differential item functioning as the statistical characteristic of an item that shows the degree to which the item might be measuring different abilities for members of distinct subgroups. The framework of DIF has become an essential part of test validation methodology and the study of test fairness. Differential item functioning is typically identified using inferential DIF detection methods. Eight DIF detection methods were explained: Logistic Regression, Mantel-Haenszel, Item Response Theory, Lord's Chi-Square, Likelihood-ratio test method, b-parameter method, Simultaneous item bias tests method (SIBTEST) and the Rasch model. For each method, a number of the most widely applied software were mentioned.

show abstract

“…Many methods detect DIF (Davidson et al, 2021). This study carried out DIF detection using three methods with the item response theory (IRT) approach.…”

Section: Introductionmentioning

confidence: 99%

Differential item functioning detection in fundamental physics test

Saepuzaman

Istiyono²,

Haryanto³

2022

CJES

View full text Add to dashboard Cite

The study's objective was to identify the differential item functioning (DIF) or bias using item response theory. This research is exploratory research with a quantitative approach. In this study, the data used is a fundamental physics test, specifically motion one and two dimensions, dynamics and work and energy concept, with 25 items in the two-tier multiple choices in form. Two hundred and fifty-four prospective physics teachers at West Java and Banten provinces, Indonesia, consisting of 103 males (reference group) and 151 females (focal group). The data analysis includes two stages: conformity of the logistic model parameters and item characteristics, followed by DIF detection. The model suitability test shows the appropriate parameter is 2PL. DIF detection has three methods: simple area indices, Raju and the Lord method. Research results show two things. First, the model fit test shows that the data best fit the 2-parameter or 2-PL logistic parameter model, and all items have good characteristics. Second, the most significant item containing uniform DIF was number 5, followed by nonuniform DIF items 9 and 25. These three items were biased because the chances of answering correctly for male and female students differed. Significantly, this means that this item favors one group. This study illustrates the importance of constructing instrument items so that all test takers, both male and female, have an equal chance to answer the question correctly. Keywords: Differential item functioning, physics test, simple area indices, Raju method, Lord method;

show abstract

Investigating Item Bias in a CS1 Exam with Differential Item Functioning

Cited by 5 publications

References 25 publications

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

Differential Item Functioning Detection Methods: An Overview

Differential item functioning detection in fundamental physics test

Contact Info

Product

Resources

About