Research project assessments and supervisor marking: maintaining academic rigour through robust reconciliation processes

McQuade, R; Kometa, Simon; Brown, Jonathon D.; Bevitt, Debra J.; Hall, Judith G.

doi:10.1080/02602938.2020.1726284

Cited by 3 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This would also help second and third assessors not succumb to the pressure upon them that is caused by the supervisor's bias. Indeed, supervisor bias is one of the factors affecting fairness and equity in marking practices [14,22]. During marking, the supervisors' knowledge of their students' background put the second assessors in an unfair position as these would mark based on what was presented to them.…”

Section: Discussionmentioning

confidence: 99%

“…In their study, [14] found that some early career supervisors inflated their marks, fearing that awarding low marks could affect their careers as it will be seen as a failure to educate and inspire students. Citing literature, [14] list the factors that influence supervisors' marks as the length of student contact, the supervisor wanting to protect the studentteacher relationship and a reluctance to award a failing grade. However, their study also showed there exists an 'internal' marker bias even though the internal markers do not know the students.…”

Section: A Marking Practicesmentioning

confidence: 99%

See 1 more Smart Citation

I betrayed my ethical principles: Investigating master’s dissertation marking practices in CS

Ada

2022

2022 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)

View full text Add to dashboard Cite

The project or dissertation is the most significant element in a master's program. In the UK, this assessment is worth a third or more credits out of 180 and contributes to student degrees, which is seen as a benchmark for employability. Despite the increased regulation and accountability regarding academic standards, there is very little debate on grading practices, particularly dissertation marking. The limited research on the assessment of master's dissertations indicates the need for more research in that area. This paper presents a study that investigated markers' (n=31) master's project dissertations marking practices in Computing Science at a university. Kruskal-Wallis tests revealed that participants' marking practices were not statistically different despite having different marking loads and master's dissertation marking experiences. Findings also show that many assessors do not typically use marking schemes; the marking load negatively affects their marking. Most would prefer complete anonymity of marking, including blind negotiation. A third of assessors have had issues marking dissertations outside their area of expertise. Thematic analysis of the qualitative data corroborated those findings and revealed three overarching themes: 1) I felt that I betrayed my ethical principles, 2) I tried to acknowledge my bias/familiarity with the topic, and 3) Improving the marking experience, second marking and reconciliation process. The paper highlights four key points to consider to improve the master's dissertation marking process.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Marking Practicesmentioning

confidence: 99%

I betrayed my ethical principles: Investigating master’s dissertation marking practices in CS

Ada

2022

2022 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)

View full text Add to dashboard Cite

show abstract

“…To ensure intra-rater reliability, it is essential to clearly define each criterion to prevent multiple interpretations by examiners. Additionally, examiners should be provided with bias-reduction training (Wylie and Szpara, 2004) to make them aware of potential biases, such as supervisor bias (Bettany-Saltikov et al ., 2009; McQuade et al ., 2020; Nyamapfene, 2012), and to take actions to prevent them. During the grading process, examiners should also consistently revisit the established criteria and level descriptors to maintain consistency.…”

Section: Problems and Guidelines In Meeting The Four Standardsmentioning

confidence: 99%

Ensuring bachelor’s thesis assessment quality: a case study at one Dutch research university

Hsiao

Watering

Heitbrink

et al. 2023

HEED

View full text Add to dashboard Cite

PurposeIn the Netherlands, thesis assessment quality is a growing concern for the national accreditation organization due to increasing student numbers and supervisor workload. However, the accreditation framework lacks guidance on how to meet quality standards. This study aims to address these issues by sharing our experience, identifying problems and proposing guidelines for quality assurance for a thesis assessment system.Design/methodology/approachThis study has two parts. The first part is a narrative literature review conducted to derive guidelines for thesis assessment based on observations made at four Dutch universities. The second part is a case study conducted in one bachelor’s psychology-related program, where the assessment practitioners and the vice program director analyzed the assessment documents based on the guidelines developed from the literature review.FindingsThe findings of this study include a list of guidelines based on the four standards. The case study results showed that the program meets most of the guidelines, as it has a comprehensive set of thesis learning outcomes, peer coaching for novice supervisors, clear and complete assessment information and procedures for both examiners and students, and a concise assessment form.Originality/valueThis study is original in that it demonstrates how to holistically ensure the quality of thesis assessments by considering the context of the program and paying more attention to validity (e.g. program curriculum and assessment design), transparency (e.g. integrating assessment into the supervision process) and the assessment expertise of teaching staff.

show abstract

How is theory used in assessment and feedback research? A critical review

Nieminen

Bearman

Tai

2022

Assessment & Evaluation in Higher Education

View full text Add to dashboard Cite

Research project assessments and supervisor marking: maintaining academic rigour through robust reconciliation processes

Cited by 3 publications

References 20 publications

I betrayed my ethical principles: Investigating master’s dissertation marking practices in CS

I betrayed my ethical principles: Investigating master’s dissertation marking practices in CS

Ensuring bachelor’s thesis assessment quality: a case study at one Dutch research university

How is theory used in assessment and feedback research? A critical review

Contact Info

Product

Resources

About