Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory

Yasuda, Jun-ichiro; Hull, Michael M.; Mae, Naohiro

doi:10.1103/physrevphyseducres.18.010112

Cited by 11 publications

(7 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, the estimates from CAT are equivalent to the results from paper-pencil. This is consistent with recent studies on the equivalence of CAT (Abidin et al, 2019;Demir & French, 2021, Yasuda et al, 2022. In addition, the implementation of CAT increased the practicality compared to the fixed-length test version by reducing test length and time.…”

Section: Discussionsupporting

confidence: 90%

“…In addition, the implementation of CAT increased the practicality compared to the fixed-length test version by reducing test length and time. Similar studies support the findings regarding the advantage of CAT in terms of test length and duration (Abidin et al, 2019;Alkhadher et al, 1998;Betz & Turner, 2011;Choi et al, 2010;Demir & French, 2021;Jodoin et al, 2006;Kezer, 2013;Paap et al, 2019;Rezaie & Golshan, 2015;Yasuda et al, 2022;Weiss, 2011).…”

Section: Discussionsupporting

confidence: 82%

“…Among the CAT studies on psychological traits, most of the literature is about simulation studies (Betz & Turner, 2011;Fliege et al, 2005;Gardner et al, 2004;Gibbons et al, 2008;Gibbons et al, 2012;Hol et al, 2007;Smits et al, 2011). On the other hand, live application studies are rare (Achtyes et al, 2015;Baek, 1993;Smits et al, 2011;Simms & Clark, 2005;Yasuda et al, 2022).…”

Section: Example Of Iccc (Right) and Cbcf (Left) For A 5-point Likert...mentioning

confidence: 99%

See 2 more Smart Citations

Applicability and Efficiency of a Polytomous IRT-Based Computerized Adaptive Test for Measuring Psychological Traits

Şimşek

TAVŞANCIL>

2022

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

Currently, research on computerized adaptive testing (CAT) focuses mainly on dichotomous items and cognitive traits (achievement, aptitude, etc.). However, polytomous IRT-based CAT is a promising research area for measuring psychological traits that has attracted much attention. The main purpose of this study is to test the practicality of the polytomous IRT-based CAT and its equivalence with the paper-pencil version. Data were collected from 1449 high school students (45% female), using the paper-pencil version for IRT parameter estimates and CAT simulation studies. For the equivalence study, the research group consisted of 81 students (47% female) who participated in both the paper-pencil and live CAT application. The paper-pencil version of vocational interest inventory consists of 17 factors and 164 items. The simulation study showed that the EAP estimation method and the SE < .500 test termination strategy were superior compared to the other CAT designs. The Item selection did not help to reduce test duration or increase measurement accuracy. As a result, it was found that an area of interest can be assessed with about four items. The results of the live CAT application showed that the estimates of CAT were strongly positively correlated with its paper-pencil version. In addition, the live CAT application increased usability compared to the fixed-length test version by reducing test length by 50% and time by 77%. This study shows that the polytomous IRT-based CAT is applicable and efficient for measuring psychological traits.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Discussionsupporting

confidence: 82%

Section: Example Of Iccc (Right) and Cbcf (Left) For A 5-point Likert...mentioning

confidence: 99%

See 1 more Smart Citation

Applicability and Efficiency of a Polytomous IRT-Based Computerized Adaptive Test for Measuring Psychological Traits

Şimşek

TAVŞANCIL>

2022

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

show abstract

“…An increased item bank for the FCI is desired, and subquestions can serve this purpose as substitutes for the original FCI items. In particular, we can integrate the subquestions we created into the item pool used by the computerized adaptive testing environment for the FCI (FCI-CAT) [17], [20]. In CAT, student ability is estimated with fewer questions administered (and hence in a shorter amount of time) by updating the ability estimate with each item that the student completes and not administering items that are too easy or too difficult for the respondent.…”

Section: Girep-2022 Journal Of Physics: Conference Series 2750 (2024)...mentioning

confidence: 99%

Methodological suggestions for creating subquestions for the Force Concept Inventory (and other research-based assessments)

Hull,

Yasuda,

Mae

2024

J. Phys.: Conf. Ser.

Self Cite

View full text Add to dashboard Cite

The Force Concept Inventory (FCI) is a research-based assessment that is used internationally to assess student understanding of Newtonian mechanics. The assessment has been investigated from a number of perspectives and many suggestions have been made for its improvement. One suggestion is the creation of subquestions. Subquestions are a tool to reduce false positives (answering a survey item correctly without correct understanding) and false negatives (answering incorrectly despite correct understanding). In this presentation, we will discuss how we created subquestions for two items on the FCI, informed by survey-based interviews with students and the original intended targets of the items. Our research aims are not only to present the subquestions we developed, but to share our methodology of creating these subquestions, as this methodology can be used in general for producing subquestions on other research-based assessments as well.

show abstract

“…The difference can be attributed to the use of the online exam as part of the evaluation process or not. Some authors [ 38 , 39 ] proposed strategies to improve test security and efficiency.…”

Section: Introductionmentioning

confidence: 99%

Active methods in electricity and magnetism courses: Influence of degree, academic level and gender on student performance

Serrano,

Vidaurre,

Meseguer-Dueñas

et al. 2023

Heliyon

View full text Add to dashboard Cite

Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory

Cited by 11 publications

References 68 publications

Applicability and Efficiency of a Polytomous IRT-Based Computerized Adaptive Test for Measuring Psychological Traits

Applicability and Efficiency of a Polytomous IRT-Based Computerized Adaptive Test for Measuring Psychological Traits

Methodological suggestions for creating subquestions for the Force Concept Inventory (and other research-based assessments)

Active methods in electricity and magnetism courses: Influence of degree, academic level and gender on student performance

Contact Info

Product

Resources

About