Investigating the medium effect in computer‐aided assessment of school Chemistry and college Computing national examinations

Abstract

This article presents results of a comparison between paper and computer tests of ability in Chemistry and Computing. A statistical model is employed to analyse the experimental data from almost 200 candidates. It is shown that there is no medium effect when specific traditional paper examinations in Chemistry and Computing are transferred into electronic format. The effect of rewording for computer-delivered test questions is also investigated and again the conclusion is that no evidence of a difference could be found. These results were obtained as part of the Project for Assessments in Scotland using Information Technology (PASS-IT).

Paper-based versus computer-based testing in engineering education

“…However this difference did not affect the test results significantly. This result supports findings of Ashton [14] which they could not find any evidence of either medium or rewording effect on test mode.…”

“…The results of this study supports the results of several studies which have found no significant difference on students' scores in CBT and P&P testing environments, [6]- [14], and [28].…”

“…For example, there are some studies showing that there is no significant performance difference between P&P tests and CBT [6]- [14]. Among those studies, Ashton et al [14] have analyzed the medium effect (the screen dumps of the computerized version vs. computerized version of paper test) and rewording effect (original paper test vs. computerized version of paper test). They could not find any evidence of either medium effect or rewording effect.…”

An investigation of the validity of course‐based online assessment methods: The role of computer‐related attitudes and assessment mode preferences

“…However, the proctored nature of the assessments should be noted here, whereas the present study is particularly concerned with unproctored online contexts, such as was used by Hewson et al (2007), which have been understudied to date. Other authors have also noted a lack of modality effects when comparing proctored, computerized course-based assessments with pen and paper administrations (e.g., Ashton, Beevers, Korabinski, & Youngson, 2005;Cassady & Gridley, 2005). Kingston (2009) reviewed 81 studies (all of which seemed to have used a proctored in-class setting), comparing computer-and paperadministered MCQ tests, in schools rather than higher education settings, and concluded that only very small effect sizes have been observed in these studies overall.…”

Use of Online Assessment to Enhance Teaching and Learning: The PASS-IT Project

“…As part of PASS-IT the effect of the delivery medium was investigated in the unit assessments in other subject areas -those of Higher Chemistry and Higher National Certificate (HNC) in Computing. Performance data was gathered from 137 Higher Chemistry students, and 44 Higher National Computing students, and the statistical analysis of this data was carried out using a general linear model, taking into account factors such as differences in ability, school and gender -more details on the statistical analysis can be found in Ashton et al (2005).…”