Bones, boys, bombs and booze: an exploratory study of the reliability of marking dissertations across disciplines

Grade Correspondence between Internal andExternal Examiners of Occupational Therapy Students’ Bachelor Theses

“…The internal examiners may use a more interpretative and holistic approach in their judgement, taking into account also their knowledge about the student's learning process and amount of effort given to the work with the thesis. This hypothesis corresponds with the results from a study evaluating the reliability of generic assessment criteria when used by lecturers from different disciplines (Bettany-Saltikov et al, 2009). In that study, there was a general agreement among the participants that marking is affected by the relationship to the student.…”

“…However, and in contrast to the results from our study, the participants argued that this can work in both a positive and negative manner. Interestingly, Bettany-Saltikov and colleagues noted that in those cases where the second examiner had marked the thesis significanty lower than the supervisor, the participants in the study agreed with the supervisors' marks (Bettany-Saltikov et al, 2009). Even if the agreement between examiners in general was high in our study, there was a discrepancy of two levels in nine cases and four levels in one case. This suggests that a more detailed set of standards descriptors for each grade may be needed.…”

“…Hence, the question of correspondence between examiners arises. Although evidence is somewhat mixed, research has generally suggested challenges related to establishing inter-examiner agreement on student work in higher education, particularly with regard to essays and other forms of complex academic student work, such as bachelor theses (Asmyhr, 2011;Bettany-Saltikov, Kilinc, & Stow, 2009;Bjølseth, Havnes, & Lauvås, 2011;Larsen, Johnsen, & Pallesen, 2006;Lauvås & Jakobsen, 2002;Rasch & Eriksen, 2008). When studying correspondence between internal and external examiners on bachelor students' essays at a Norwegian university college, Asmyhr (2011) uncovered differences between examiners that surpassed two levels on a six-level grading scale.…”

An evaluation of the assessment tool used for extensive mini-dissertations in the Master's Degree in Family Medicine, University of the Free State

“…After attending training workshops, the deviation between assessments decreased from 3.89 to 0.51. The importance of training on assessment tools and staff development was also emphasised by Bettanny-Saltokov, Kilinc and Stow, 13 Hand and Clewes 17 and Matthews. 14 Question 4 focused on the improvement of the current assessment tool.…”

“…A study by Webster, Pepper and Jenkins 16 demonstrated the difference in overall opinion (global score) and marks allocated to a research project. Bettany-Saltikov, Kilinc and Stow 13 concluded that it is best to use analytic and holistic marking combined, when assessing dissertations. Thus, the use of assessment category marks, as well as a global assessment, as proposed during the focus group interview, is supported by the literature.…”