The Impact of Item Stem Format on the Dimensional Structure of Mathematics Assessments

Kan, Adnan; Bulut, Okan; Cormier, Damien C.

doi:10.1080/10627197.2018.1545569

Cited by 15 publications

(15 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tests that rely heavily on written text require test-takers to have competencies, such as verbal, reading comprehension, and interpretation skills, that are often irrelevant to the construct that the test purports to capture (Popp et al, 2016). Therefore, the extensive use of text in assessments that do not measure language skills is problematic, especially for tests that are administered to test-takers with varying levels of reading comprehension or poor proficiency in the language of the test (Kan et al, 2018). Indeed, research has demonstrated that the amount and complexity of written text in tests can negatively impact the performance of certain groups of test-takers (Abedi, 2004(Abedi, , 2006.…”

Section: Introductionmentioning

confidence: 99%

Animated videos in assessment: comparing validity evidence from and test-takers’ reactions to an animated and a text-based situational judgment test

Karakolidis

O’Leary

Scully

2021

International Journal of Testing

View full text Add to dashboard Cite

The linguistic complexity of many text-based tests can be a source of construct-irrelevant variance, as test-takers' performance may be affected by factors that are beyond the focus of the assessment itself, such as reading comprehension skills. This experimental study examined the extent to which the use of animated videos, as opposed to written text, could (i) reduce construct-irrelevant variance attributed to language and reading skills and (ii) impact test-takers' reactions to a situational judgment test. The results indicated that the variance attributed to construct-irrelevant factors was lower by 9.5% in the animated version of the test. In addition, those who took the animated test perceived it to be more valid, fair, and enjoyable, than those who took the text-based test. They also rated the language used as less difficult to understand. The implications of these findings are discussed.

show abstract

Section: Introductionmentioning

confidence: 99%

Animated videos in assessment: comparing validity evidence from and test-takers’ reactions to an animated and a text-based situational judgment test

Karakolidis

O’Leary

Scully

2021

International Journal of Testing

View full text Add to dashboard Cite

show abstract

“…There is a bank of questions that instructors can utilize to compile their assessment, therefore saving them time. Moreover, items designed to be auto-graded usually do not require a high level of reading comprehension (Kan et al, 2019). High-stakes assessments, such as SATs or GCSEs, use E-assessments exclusively.…”

Section: Benefits Of E-assessmentmentioning

confidence: 99%

“…First, the lack of technical skills, limitations of access, and poor infrastructure might affect the assessment process (Alruwais et al, 2018). In addition, the items' format has a substantial influence (Kan et al, 2019). The use of MCQ is widespread in the E-assessment process in mathematics.…”

Section: Challenges Of E-assessmentsmentioning

confidence: 99%

The COVID-19 Pandemic and the Challenges of E-Assessment of Calculus Courses in Higher Education: A Case Study in Saudi Arabia

Azmi

Khoshaim

2021

IJLTER

View full text Add to dashboard Cite

The COVID-19 pandemic has affected many aspects of our lives, including education. Due to this unexpected catastrophe, education has shifted to virtual-learning and auto-grading models in most parts of the world. This study explores the validity and appropriateness of auto-grading-assessment for online exams by comparing students’ online exam scores where they are first auto-graded and then manually graded. Furthermore, it investigates whether the mean differences in their scores are statistically significant. The study included two calculus courses taught by the authors, during the spring semester 2019-2020 at a private university in Saudi Arabia. The online exam was performed on the WebAssign platform, which has built-in calculus questions. The sample consisted of fifty-five students who were registered on those calculus courses. The quantitative data was analysed using the SPSS statistical tool. A paired t-test at an alpha level of 0.05 was performed on differences in mean exam scores between auto-graded and manually-graded scores. The statistical analysis results revealed a statistically significant difference in students' mean scores. Our findings illustrate the importance of human intelligence, its role in assessing students' achievements and understanding of mathematical concepts, and the extent to which instructors can currently rely on auto-grading. A careful manual investigation of auto-graded exams revealed different types of mistakes committed by students. Those mistakes were characterized into two categories: non-mathematical mistakes (related to Platform Design) and minor mathematical mistakes, which might deserve partial credit. The study indicated a need to reform the auto-grading system and provided some suggestions to overcome its setbacks.

show abstract

“…Based on the test results, students' reasoning abilities are high, medium and low. Highly skilled students can analyze mathematical situations using patterns and relationships appropriately, and systematic work steps, can formulate valid arguments and be able to draw logical conclusions (Kan et al, 2019). Students with moderate abilities can analyze mathematical situations using patterns and relationships appropriately, and unsystematic steps, so they are less able to formulate valid arguments and are less able to draw logical conclusions (Alexander et al, 2016).…”

Section: Figure 3 Answers To Low Proficiency Subjectsmentioning

confidence: 99%

“…The errors experienced by students in solving story problems in the aspect of understanding the problem are errors in interpreting language and mathematical models (Kan et al, 2019). The causes of errors in the aspect of understanding language are students' low mathematical abilities and incomplete/wrong student reasoning.…”

Section: Figure 3 Answers To Low Proficiency Subjectsmentioning

confidence: 99%

Analysis of Students Mathematical Reasoning Abilities on Number Topics

al.¹

2021

pae

View full text Add to dashboard Cite

Mathematical reasoning skills are critical for students to solve math problems. The research objective was to determine the reasoning ability of junior high school students in solving number problems. This research is a qualitative descriptive study with case studies. The research subjects were 27 students consisting of 14 female students and 13 male students. A total of 10 questions were tested containing five multiple-choice questions, and five description questions. From the results of data analysis, 3% of students had high reasoning abilities, 43% of students had moderate reasoning abilities, 30% of students had low reasoning abilities, 13% of students had meagre reasoning abilities.

show abstract

The Impact of Item Stem Format on the Dimensional Structure of Mathematics Assessments

Cited by 15 publications

References 68 publications

Animated videos in assessment: comparing validity evidence from and test-takers’ reactions to an animated and a text-based situational judgment test

Animated videos in assessment: comparing validity evidence from and test-takers’ reactions to an animated and a text-based situational judgment test

The COVID-19 Pandemic and the Challenges of E-Assessment of Calculus Courses in Higher Education: A Case Study in Saudi Arabia

Analysis of Students Mathematical Reasoning Abilities on Number Topics

Contact Info

Product

Resources

About