Process mining based approach to performance evaluation in computer‐aided examinations

Abstract

A considerable part of educational systems tends to be online and computer oriented. However, online examination may create some difficulties during the evaluation of student performance. Process mining which arises as a new concept presents various powerful techniques for processing and analyzing different types of data by making use of some advanced information technologies. This paper proposes a novel approach based on process mining for evaluating the performance of students that should follow certain tasks on the computer. The proposed approach is composed of two main phases which are process mining and similarity analysis. Automatic assessment is performed totally in six steps in order to obtain students’ final grades. In addition, cheat control is possible in the last step thanks to the similarity analysis. A real‐life application in an Enterprise Resource Planning (ERP) course is performed in order to present usefulness, validity and practicality of the proposed approach. Furthermore, to evaluate the performance of the assessment system, we compared the assessment mechanism against instructor. A total of 15 students’ answers belonging to computer‐aided exam are assessed by instructor and the results showed a very good agreement between the automatic assessment system and instructor.

A Systematic Review of Online Exams Solutions in E-learning: Techniques, Tools and Global Adoption

“…35 [53], [41], [55], [42], [43], [57], [28], [14], [29], [30], [58], [45], [31], [46], [32], [19], [33], [65], [15], [66], [59], [34], [48], [35] Table 6: Leading online exam features targeted in the selected studies…”

“…and Evaluation [43], [56], [30], [58], [63], [46], [32], [65], [66], [59] 4 Usability [57], [15] 2 5 Other [20], [13], [52], [27], [44], [29], [34], [60], [48] 9…”

“…Hardware / Software Virtualization Technique [20] 9 Time Adaptive for Mobile E-Exam (TAMEx) Technique [37] 10 Fuzzy Clustering Technique [23] 11 Two dimensional Gesture Detection Technique [24] 12 K Means Clustering and Rule Mining Algorithms [41] 13 Bayesian Network based Technique [30] 14 Quantified Event Automata and π-calculus Technique [31] 15 Dynamic Profile Questions Technique [19] 16 Syntactical Relational Feature Extraction Technique [46] [50], [61], [53], [55], [42] 2 PHP [21], [51], [29], [30], [58] 3 Java [27], [ 14 MySQL [21], [51], [29], [58] 15 FireBase [36], [51], [55] 16 SQLlite Databases / Storage [50], [53] 17 SQL Server [27], [64] 18 JSON [36] 19 Hidden Camera Activity 12 Image Capturing [36] 20 Face++ 13 Facial Recognition Platform [27] 21 WeScan 14 Documents Scanning [55] 22 Bayesian Network tools in Java (BNJ) 15 Probability Models Toolkit [30] 23 ProVerif 16 Formal Verification Tool [31] 24 Disco 17 process mining toolkit [48] 25 Weka 18 Machine Learning Tool [35]…”

Efficacy and perceptions of assessment digitization within a large‐enrollment mechanical and aerospace engineering course

“…Trends of increasing enrollment, reduction in costs of PCs, and the success of computerized testing in other disciplines have been motivating its recent research in Engineering . For instance, the University of Utah and Brigham Young University (BYU) Testing Centers (TCs) recently provide computerized testing facilities for engineering and science programs using digitized assessments.…”

“…Unfortunately, diligent efforts by the instructor and Graduate Teaching Assistants (GTAs) to realize accurate paper‐based assessments contribute negligible transferable progress towards those same tasks in subsequent semesters. The feasibility of digitized exams within engineering disciplines has received increasing attention in recent years . Challenges for assessment digitization within engineering curricula include partial credit, solution composability/traceability via handwritten work, and assessment of problem solving aspects within the constraints of contemporary Learning Management Systems (LMSs) .…”