Interactive PDF Forms to Conduct Online Examinations in University Education: Practical Experience and Lessons Learned

Abstract

Due to the Corona Pandemic, paper-based examinations had to be transferred to an online format on short notice. At the Lucerne University of Applied Sciences and Arts we conducted three online examinations with a total of 816 students using an interactive PDF form. We faced two major challenges redesigning our examinations: 1) avoiding unnecessary stress to students' working memory by choosing a simple examination design and 2) minimizing the possibility of cheating. In our paper we explain how we addressed these challenges working with an interactive PDF form in combination with the learning management system Ilias. Our results show that using an interactive PDF form lowers the extraneous cognitive load on our students, increases our efficiency in correcting, reduces error rates in grading and further allows for faster feedback to our students.

High-Stakes Online Exams

“…This included technical issues such as system spikes, compatibility issues (operating systems, versions, processing speed), administrative issues (exam scheduling with additional retake slots in case of system failure), fairness vis-à-vis students in exam rooms with better connectivity, and specific application issues (formatting in Word v PDF, media size in PPT v PDF, and so on). This led most faculty to choose solutions based on "affordances" to improve exam security: Not always the optimal solution, but rather the safest, easiest or most convenient (see [3] for this and for more detailed information on the use of PDFs in exams).…”

“…While exam grading practice has grown over the years with this in mind, most studies have focused on the number of raters involved and the number of criteria assessed [33], [41]. This leads to safeguards assuring exam reliability by identifying item discriminators while simultaneously increasing faculty cognitive offloading [3]. It is our contention that online exam design will favor items that discriminate, such as multiple choice, over higher taxonomy items with high variance, as the latter would lower interrater reliability.…”

“…Alghamdi et al [2] propose an intelligent e-learning system for assessment with the goal to reduce the need for proctoring. While such AI-based tools can certainly increase efficiency to a certain extent, the logistics of ensuring exam quality can prove quite challenging [3] and can even prolong the process. Still other have focused on the central question of academic integrity in exams and unethical decision-making [4], the benefits of group exams on the learning experience [5] and the use Bloom's taxonomy to raise the cognitive demand on exams [6], [7], [8].…”

“…However, any efforts to increase exam integrity would also lead to a greater potential for system failure. Therefore, faculty in procedural subjects moved away from "showing your work" to answer sheets that decreased complexity and increased security in the following ways: 1) the use of widespread applications such as Adobe PDF [3]; 2) multiple-choice exams with up to ten possible answers, nearly eliminating random guessing; 3) staggered exam delivery with question items in binary up to 25 digits (e.g. instead of "question 4", 101110001011011010111001), thus severely complicating student collusion.…”

Perceived Nexus Between Non-Invigilated Summative Assessment and Mental Health Difficulties: A Cross Sectional Studies

“…Test anxiety among students have been studied in both non-invigilated online and traditional modes of examinations (Lowe, 2019;Alibak et al, 2019;Mastour et al, 2021). While the literature is not concrete on which type of assessment-online or traditionalcauses more anxiety, there is evidence to suggest that students experience a certain level of test anxiety with all forms of assessments (Harley et al, 2021;Keimer et al, 2022). Proponents of online assessment report that it reduces overall test anxiety (Alderbashi, 2021).…”