External examining: fit for purpose?

Abstract

Usage of any items from the University of Cumbria's institutional repository 'Insight' must conform to the following fair usage guidelines.Any item and its associated metadata held in the University of Cumbria's institutional repository Insight (unless stated otherwise on the metadata record) may be copied, displayed or performed, and stored in line with the JISC fair dealing guidelines (available here) for educational and not-for-profit activities provided that• the authors, title and full bibliographic details of the item are cited clearly when any part of the work is referred to verbally or in the written form• a hyperlink/URL to the original Insight record of that item is included in any citations of the work • the content is not changed in any way• all files required for usage of the item are kept together with the main item file. You may not• sell any part of an item• refer to any part of an item without citation • amend any item or contextualise it in a way that will impugn the creator's reputation• remove or alter the copyright statement on an item. AbstractIn a context of international concern about academic standards, the practice of external examining is widely admired for its role in defending standards. Yet a contradiction exists between this faith in examining and continuing concerns about standards. This article argues that external examining rests on assumptions about standards which are significantly open to challenge. Six assumptions relating to the conceptual context, the operation and the nature of examiners themselves are analysed drawing on a review of the available evidence. The analysis challenges the notion of a consensus on standards and the potential to vest in individuals the ability to represent that consensus when judging the comparability of academic standards in a stable and appropriate way.The issues raised have relevance to the UK and to other national systems using external examiners or seeking to guarantee academic standards by, in some cases, adopting quality assurance approaches developed in the UK. 2 External examining: fit for purpose?External examining is a process by which the fairness of assessment and the academic standards in higher education are monitored. This monitoring normally involves an academic from another institution approving assessment tasks and examinations and checking a sample of student work for the courses which they examine. A wider range of activities may also be included as discussed below. In certain limited circumstances, for example in viva voces for research degrees, external examiners directly examine student performance. In the case of vocational courses, some examiners may be practising members of the relevant profession rather than academics. External examining is prevalent in the UK and several other countries operate or have operated an external examiner system:

Examining the assessment literacy of external examiners

“…While the drive towards standardization across the system has resulted in a great deal of focus on the reliability, comparability, and transparency of procedures, far less attention has been paid to the quality of the underlying practice (Bloxham, 2009). As identified in the introduction, of the research that does exist, the findings do not inspire confidence and point to a system that is underpinned by largely unchallenged and unsupported assumptions (Bloxham and Price, 2015). One such assumption revolves around the assessment literacy of external examiners, which has provided the focus of this study.…”

“…The aim of this study is to evaluate the extent of assessment literacy demonstrated within the written reports of external examiners, thereby testing the validity of the unchallenged assumption of its presence within the external examining system (Bloxham and Price, 2015).…”

“…External scrutiny of higher education (HE) courses is evident globally, including but not restricted to New Zealand, South Africa, Denmark, Malaysia, India, and the UK (Bloxham and Price, 2015). The use of an impartial peer reviewer, external to the host institution for the purpose of quality assurance (referred to henceforth as an external examiner), has been a distinguishing feature of UK HE since the 1830s (Silver, 1996).…”

“…Indeed, of the research that does exist, the findings do not inspire confidence (Silver et al, 1995;Cuthbert, 2003;Orr, 2007). For example, issues relating to a range of aspects underpinning the role include the effectiveness (Silver et al, 1995;Elton, 1982); impact (Murphy, 2006); reliability (Price, 2005); consistency (Brooks, 2012;Cooke, 2002); comparability and quality (Bloxham and Price, 2015;Bloxham, 2009); difficulty in codifying and assuring academic standards (Bloxham et al, in press;Sadler, 2014); variation in interpretation of standards-related language and lack of inter-examiner agreement (Bloxham et al, in press); as well as the restrictive impact on innovation of assessment practices (Biggs, 2003). In essence, the focus of external evaluations tends to overlook the substance of what is being evaluated in favour of the constituent processes (Harvey, 2002).…”

Assessment in Higher Education: The Potential for a Community of Practice to Improve Inter-marker Reliability

“…In recent years, both the concept and application of learning outcomes have received much attention in the literature (Bloxham and Price, 2013;QAA and HEA, 2013), with curriculum and assessment design being the main consideration and relatively less attention being given to how ILOs might actually connect with the practical issues involved in scoring students' individual achievement. However, interest in this issue is increasing in both the academic literature and the popular press.…”