The influence of test mode and visuospatial ability on mathematics assessment performance

Abstract

Mathematics assessment and testing are increasingly situated within digital environments with international tests moving to computer-based testing in the near future. This paper reports on a secondary data analysis which explored the influence the mode of assessment-computer-based (CBT) and pencil-and-paper based (PPT)-and visuospatial ability had on students' mathematics test performance. Data from 804 grade 6 Singaporean students were analysed using the knowledge discovery in data design. The results revealed statistically significant differences between performance on CBT and PPT test modes across content areas concerning whole number algebraic patterns and data and chance. However, there were no performance differences for content areas related to spatial arrangements geometric measurement or other number. There were also statistically significant differences in performance between those students who possess higher levels of visuospatial ability compared to those with lower levels across all six content areas. Implications include careful consideration for the comparability of CBT and PPT testing and the need for increased attention to the role of visuospatial reasoning in student's mathematics reasoning.

Spatial Skills, Reasoning, and Mathematics

“…High school students' performance on the MRT has been found to relate to both geometry course grades and performance on a geometry achievement test, as well as to the students' perceptions of how well they do in geometry (Weckbacher & Okamoto, 2014); MR was also positively related to the students' perceptions of how well they do in algebra, but not to their actual algebra course grades (Weckbacher & Okamoto, 2014). One recent study, however, found that scores on a paper folding task were predictive of both algebra pattern knowledge and geometry problem-solving for 6 th grade students in Singapore (Logan, 2015).…”

Establishing Measurement Equivalence Across Computer- and Paper-Based Tests of Spatial Cognition

“…First, both construct and score equivalency of test medium is necessary if test scores are to be comparable among different studies. Second, measurement equivalence among test types is needed to adequately report changes over time as tests are converted from paper-and-pencil to computer format (see also Logan, 2015). Finally, researchers must establish equivalency of tests to be certain that abilities and constructs are measured in the same way for both paper-based and computerized testing.…”

“…In the past decade, the computerized assessment of many cognitive abilities has become seemingly ubiquitous, and computer-based testing is assumed to be an acceptable replacement for more traditional, paper-based performance measures (Jamil, Tariq, & Shami, 2012;Leeson, 2006). One reason for this transition is the many advantages offered by computer-based testing, including automated scoring, rapid administration, use of fewer resources (e.g., paper, pencils, test administrators), and potential for adaptive testing (Arce-Ferrer & Guzman, 2009;Csapo, Molnar, & Nagy, 2014;Leeson, 2006;Logan, 2015;McDonald, 2002;Williamson, Williamson, & Hinze, 2017). Additionally, participants tend to favor computerized test administration over paper-and-pencil testing (Vispoel, 2000;Vispoel, Boo, & Bleiler, 2001).…”

“…Beyond the design limitations, published studies have used "various, often disparate, psychometric analyses, making comparison difficult" (Gates & Kochan, 2015, p. 168). For example, in the past 5 years, research comparing test administration has been published with analyses such as parametric and nonparametric mean comparisons, regression, difficulty and discrimination analyses, and, less often, tests of measurement equivalence (Csapo et al, 2014;Fazeli, Ross, Vance, & Ball, 2013;Logan, 2015;Williamson et al, 2017).…”

Performance Assessment on High School Advanced Algebra

“…Educational research in this specific topic has been conducted recently from different perspectives. Examples include positive biases in self-assessment of mathematics competence, achievement goals, and mathematics performance (Dupeyrat, Escribe, Huet, & Regner, 2011); the use of technology for testing students' mathematics performance (Logan, 2015); students' challenges while demonstrating mathematics performance and understanding through oral testing (Ianonne & Simpson, 2015); the relationship between students' performance on conventional assessments and complex mathematical modeling problems (Kartal, Dunya, Diefes-Dux, & Zawojewski, 2016); as well as creating mathematics performance assessments for multiple student levels.…”