Evaluating a learning analytics dashboard to detect dishonest behaviours: A case study in small private online courses with academic recognition

Abstract

Background: Small private online courses (SPOCs) are one of the strategies to introduce the massive open online courses (MOOCs) within the university environment and to have these courses validates for academic credit. However, numerous researchers have highlighted that academic dishonesty is greatly facilitated by the online context in which SPOCs are offered. And while numerous algorithms have already been proposed, no research has been performed on how to transfer this information to instructors, so that they can intervene and decrease the prevalence of this issue.Objectives: In this article, we present a qualitative evaluation of a tool for detecting and monitoring students suspected of academic dishonesty in SPOCs in Selene, a Colombian instance of Open edX. Methods:The evaluation was carried out through semi-structured interviews with four instructors who taught SPOCs with academic recognition at the University of Cauca.Results: The evaluation results indicated that participants found the dashboard reliable and appropriate to detect academic dishonesty behaviours in order to intervene in these cases.Implications: But interventions are difficult to systematise, need an institutional policy, and there is uncertainty about whether these interventions can actually contribute to decreasing academic dishonesty.

Trustworthy remote assessments: A typology of pedagogical and technological strategies

“…According to Chiang et al (2022), instructors' attitudes towards academically dishonest behaviour is very important because, if they disregard dishonest actions, students will perceive that dishonest behaviour is allowed in academic contexts (and potentially in professional contexts). With the aim of supporting instructors in monitoring dishonest behaviour, Jaramillo‐Morillo et al (2022) developed a dashboard that includes two metrics obtained through an algorithm that was previously validated (Jaramillo‐Morillo et al, 2022): one related to the similarity in online exam answers, and another one related to the time closeness when submitting exam responses. Four instructors of SPOCs evaluated the use of the dashboard during the first semester of 2021, and according to information obtained through the interviews, they agreed that the metrics were adequate to obtain the required information to assess students' academic dishonesty.…”

“…According to Jaramillo‐Morillo et al (2022), the four instructors involved in their study agreed that the main intervention to prevent dishonest behaviours is to approach students who have exhibited suspicious behaviours and give them warnings about their conduct. According to one of the instructors, the first instance should be just a warning, but repeated behaviours should lead to disciplinary actions.…”

“…In the context of exams, Surahman and Wang (2022) suggest monitoring student answers, corroborate the data, and talk to students when necessary. To do so, instructors could rely on dashboards as the one proposed by Jaramillo‐Morillo et al (2022), in addition to relying on automatic warnings before and during exams. Still, Approach number 8 suggests letting instructors decide whether to issue a warning or to pursue a grade penalty, considering assessment timing throughout the academic period, low or high‐stakes, student background, etc.…”

“…As a result of the integration and triangulation of past and recent findings, our analysis and synthesis of the accepted papers came up with novel approaches to inform the design and implementation of valid and reliable assessments. Among this novel approaches, Jaramillo‐Morillo et al (2022) propose the use of a dashboard to help teaching staff members to address cheating among their students on real time, going beyond the use of machine‐learning algorithms to detect online cheating in retrospectively (Alexandron et al, 2017; Ruiperez‐Valiente et al, 2017). Along with the work of Jaramillo‐Morillo et al (2022), Maertens et al (2022) proposes the use of plagiarism tools to create shared meaning about cheating in performance‐based assessments throughout the different units of a course, expanding the existing predominant focus on online proctoring exams (Miller, 2013).…”

“…Among this novel approaches, Jaramillo‐Morillo et al (2022) propose the use of a dashboard to help teaching staff members to address cheating among their students on real time, going beyond the use of machine‐learning algorithms to detect online cheating in retrospectively (Alexandron et al, 2017; Ruiperez‐Valiente et al, 2017). Along with the work of Jaramillo‐Morillo et al (2022), Maertens et al (2022) proposes the use of plagiarism tools to create shared meaning about cheating in performance‐based assessments throughout the different units of a course, expanding the existing predominant focus on online proctoring exams (Miller, 2013). And these are only a few examples of how the studies accepted in this special issue contribute to the current knowledge.…”