Incorporating partial credit in computer‐aided assessment of Mathematics in secondary education

Abstract

In a mathematical examination on paper, partial credit is normally awarded for an answer that is not correct, but, nevertheless, contains some of the correct working. Assessment on computer normally marks an incorrect answer wrong and awards no marks. This can lead to discrepancies between marks awarded for the same examination given in the two different media. The current paper presents possible solutions to this problem and the results of experiments designed to test how successful these solutions are in practice. In light of the findings, developments to the assessment engine have been made and some questions redesigned for use in real automated examinations. The results were obtained as part of the Project for Assessments in Scotland using Information Technology (PASS-IT): a major collaborative programme involving the leading educational agencies in Scotland (see http://www.pass-it.org.uk for more details).PASS-IT has demonstrated that the computer can help measure lower order student skill profiles provided the computer assessment package is sophisticated enough. Optional steps are required to mimic partial credit; randomisation of parameters is needed for practice and the avoidance of copying; the ability to capture and mark automatically mathematical expressions and short free text responses; and the delivery in a number of feedback modes are all vital ingredients of an automatic assessment system. PASS-IT has shown how to ensure that education drives technology and not vice versa. Finally, collaboration has been paramount within PASS-IT and should continue. No single group has all the keys to unlock the future of computer-aided assessment. Scotland is well placed to move forward and eassess where its students e-learn in a large range of subjects such as those delivered via the SCHOLAR Programme (see http://scholar.hw.ac.uk). It should be done too by ensuring that teachers remain central to the learning cycle by supporting the demanding work they do through the supply of suitably filtered data on student performance. Special educational needs can also be addressed more effectively by the use of technology. BackgroundWhen changing from assessment on paper to assessment by computer, many differences may be introduced; for example, it may be necessary to reword the question for computer delivery. Alternatively, the keyboard and/or mouse must be employed to answer the question rather than a pen on paper (see . If the outcome of the assessment on paper turns out to be different from that of the examination delivered by computer, it would not be clear which (if any) of these factors was the cause. Because of the large number of factors involved in the change from paper-based testing to electronic assessment, it is necessary to separate the differences. Experiments can then be designed to determine whether any particular difference did contribute to a change in performance. One experiment undertaken in this investigation on mathematics examinations was by Fiddes et al (2002) in which the effect of the delivery m...

Computer Aided Assessment of Mathematics Using STACK

“…This kind of interaction model has been successful, and is widely used, e.g. see (Ashton et al 2006). The STACK system has been modified and extended at Aalto University, Finland and is in regular use by large groups of engineering students.…”

“…Rather than generating reliable diagnoses of a student's difficulties, they provide feedback which is designed to help students learn particular skills. For example, the group at Heriot-Watt University in the United Kingdom have more than a quarter of a century of experience in this area (see Beevers et al 1991;Ashton et al 2006). From the outset, one goal of this project was to assess students' steps in working.…”

A flexible, extensible online testing system for mathematics

“…Thus partial credit can be awarded for an answer that is not correct, but contains some of the correct working (Ashton, Beevers, Korabinski & Youngson, 2006;Darrah, Fuller & Miller, 2010).…”

E-assessment: Past, present and future

“…One of the signatures of the CALM family of assessment systems is the use of 'Steps', allowing a question to be broken into manageable steps for the benefit of students who are not able to proceed without this additional scaffolding (Beevers & Paterson 2003, Ashton et al 2006a. For the question shown in Figure 2a, a student could opt to work out the answer without intermediate assistance, and in summative use they would then be able to obtain full credit.…”