Same graph, different data: A usability study of a student-facing dashboard based on self-regulated learning theory

Abstract

Student-facing learning analytics dashboards have the potential to reconnect students with their purpose for learning, reminding them of their goals and promoting reflection about their learning journey. However, far less is known about the specifics of the relationship between different types of visualisations and data presented in dashboards and their impact on students’ motivation. In this study, we used a Human-Centred Design method across three iterations to (1) understand how students prioritise similar visualisations when presenting different data (2) examine how they interact with these, and (3) propose a dashboard design that would accommodate students’ different motivational needs. In the first iteration, 26 participants ranked their preferred visualisations using paper prototypes; in the second iteration, a digital wireframe was created based on the results from the first iteration to conduct user tests with two participants; and in the third iteration, a high-fidelity prototype was created to reflect findings from the previous iterations. Overall, findings showed that students mostly valued setting goals and monitoring their progress from a multiple goals approach, and were reluctant about comparing their performance with peers due to concerns related to promoting unproductive competition amongst peers and data privacy. Implications for educators and learning designers are discussed.

The missing link in Learning Analytics

“…The second most important component was ensuring the dashboard was easily interpreted by students which influenced our design decision to prioritise pie charts. This lines up with research conducted by De Barba et al (2022). Our implementation has demonstrated the importance of planning, as Initiative 1 was designed in parallel with the assessment running due to time pressures.…”

“…Existing research focuses on the success and engagement of SFDs (Pokhrel, J., et al, 2021)), the frameworks SFDs can be based on (Teasley 2021) and which components might best be displayed (De Barba et al, 2022). Google Scholar searches yielded a limited amount of papers ) highlighting the implementation and software of a dashboard including Quincey, E., et al, (2019).…”

“…The literature is not clear what components make up a successful SFD. De Barba et al (2022) have explored the usability of a SFDs, which focuses on "bring[ing] their attention back to the course and provide insights into their current efforts" (p. 2) with pie charts evaluated as the best approach for "performance goal for quizzes, assignments and exams" and "students prioritise looking for information that will help them evaluate how they are progressing towards their goals" (De Barba et al, 2022, p. 2).…”