Can Higher Education Exams Be Shortened? A Proposed Methodology

Abstract

Abstract-Any lecturer would agree that marking exams is the bane of her existence. A time-consuming and tiring process, it often requires complex, subjective judgments. Higher education exams typically take 3.0 hours. Do they really need to last so long? Can we justifiably reduce the number of questions on them? Shortening an exam by one hour, if justified, should result in a one-third reduction in lecturer time and effort spent marking. Surprisingly little empirical research has addressed these problems. Classical methods may be partly to blame for this dearth of studies. We propose an alternative methodology based on three key components including two recent developments in experimental design and statistics --synthetic experimental designs and equivalence hypothesis testing. The third component consists in comparing, on six psychometric criteria, student performance in a class on the standard 3.0-hr final exam with that on shortened exams with proportionately fewer questions. Two are the frequently misunderstood standard psychometric criteria -reliability and validity. We argue that adding four common-sense criteria -justifiability of test use, number of exam questions, equivalence in mean student performance, and correspondence (between shortened and full-length exam scores) -confer significant additional benefits. Our approach provides a simple methodology that lecturers can, with minimal time and effort, use to examine the effect of shortening exams for their own classes.Index Terms-Exam length, psychometric criteria, synthetic experimental designs, test length. I. INTRODUCTIONMarking exams is the bane of any lecturer's existence. It is characteristically a tedious, time-consuming process. The intricate subjective judgments that are required can exhaust even the most dedicated of lecturers. Yet despite the almost universal loathing of lecturers for this activity, surprisingly little research has been published on ways to reduce the time and effort required to mark conventional written examinations. The purpose of the present study is to redress this long-standing neglect. In this paper, we are concerned with mixed-format exams used most commonly in academe and consisting of a mixture of different types of questions including problem solving (requiring detailed solutions), Manuscript received November 6, 2013; revised January 12, 2014 essay, short answer, and multiple choice questions.Written final examinations three hours or more in length are common in many universities and colleges around the world. Why then, if marking is so universally loathed, are exams so long (both in duration and number of questions posed)? Custom or tradition seems the primary reason [1]. Even more surprisingly, long exams have been retained despite dramatic increases in class sizes over the past couple of decades at most institutions of higher learning. Many lecturers complain of class sizes more than doubling in just the last 15-20 years [2]- [4]. In effect, marking time and effort for many lecturers has, at a minimum, effective...

Analyzing the decline of student scores over time in self‐scheduled asynchronous exams

“…Running paper‐and‐pencil exams for large classes (e.g., 200+ students) presents management challenges that include requesting space, printing exams, proctoring, timely grading, and handling conflict exams (Lee, Garg, Bygrave, Mahar, & Mishra, ; Muldoon, ; Zilles et al, ). These logistic burdens discourage faculty from using pedagogies that have been shown to improve student learning, such as frequent testing (Bangert‐Drowns, Kulik, & Kulik, ; Leeming, ) and mastery learning (Kulik, Kulik, & Bangert‐Drowns, ; Pennebaker, Gosling, & Ferrell, ), and have led to the overuse of multiple‐choice exams (Scouller, ; Stanger‐Hall, ).…”

How to Avoid the Sins of Questionnaire Abridgement - guideline

“…Evaluation criteria for SFs mainly deal with proper representation of the measured construct, or a uniform allocation of items in subscales if the tool is multi-dimensional. These problems may be described by empirical criteria, the most important of which are: 1) SF reliability; 2) validity (mainly face and content validity relative to the FF); 3) the extent to which the SF reproduces the structure of the FF (this is the criterion is often used in abridgement reports); 4) similar pattern of relationships with other variables to the FF, particularly with respect to demographic variables, such as age or education, but also personal traits and characteristics (see also Lee et al, 2014).…”

“…The last stage of SF creation should consist of assessing the classification accuracy of the SF by comparing the results with classification based on the FF. Lee et al (2014) suggested a threshold of r > .90 of correlation between raw scores on the SF and FF. Ultimately the SF cannot be significantly different from FF nor should it serve another goal than the one selected by a researcher.…”

“…A distinction has also been drawn between abridgement for diagnostic purposes and abridgement for research and screening purposes (Crawford, Allan, & Jack, 1992;Bors & Stokes, 1998;Clara & Huynh, 2003;Hamel & Schmittmann, 2006). There have, however, been few attempt to produce general, theoretically based, practical guidance on how to carry out abridgement (see also Smith, McCarthy, & Anderson, 2000; for an example in the higher education field see Lee, Bygrave, Mahar, & Garg, 2014). An empirical comparison of shortening QUESTIONNAIRES ABRIDGEMENT 4 methods has not been supported by many literature sources yet (Prieto, Alonso, & Lamarca, 2003;Kleka, 2013;Ziegler, Kemper, & Kruyen, 2014a).…”