Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students

Abstract

Learning analytics dashboards (LADs) can provide learners with insights about their study progress through visualisations of the learner and learning data. Despite their potential usefulness to support learning, very few studies on LADs have considered learners’ needs and have engaged learners in the process of design and evaluation. Aligning with that, there is a limited understanding of what specific student cohorts, in particular distance and online learners, may seek from LADs to effectively support their studies. In this study, we present findings from 21 interviews with undergraduate distance learners, mainly high performers, that aimed to capture student perceptions about the usefulness of specific LAD features and the factors that explain these perceptions. Our findings revealed that amongst the LAD features favoured by students was the potential to receive study recommendations, whereas comparison with peers was amongst the least favoured elements, unless informed by qualitative information. Factors including information trust, attitudes, age, performance and academic self-confidence were found to explain these perceptions.

The Usefulness of Online Learning During the Covid19 Pandemic as Perceived by Engineering Education Students: A Multidimensional Model

“…Extant research shows various perceptions of distance education usefulness. Many studies are highlighting the advantages of distance learning in terms of learning flexibility, efficiency, and outcomes, self-efficacy, comfort (Almahasees et al, 2021;Cicha et al, 2021, Garcia-Alberti et al, 2021Fahrurrozi et al, 2020;Manea et al, 2021;Rizun & Strzelecki, 2020;Rets et al, 2021). Also, most studies emphasize an agreement on the fact that the educational created by the Coronavirus pandemic, online teaching and learning became a global solution -embraced by the whole world -that tried to cover, at least in part, in part the problems encountered by the educational systems.…”

“…At the same time, these student's opinions did not correlate positively with how students assess the ease of participating in distance learning. Rets et al (2021) presented an interview with undergraduate students that aimed to capture students' perceptions of the usefulness of Learning analytics dashboards (LADs) and the factors that explain these perceptions to students learning remotely. Identified factors: confidence in information, performance, academic self-confidence, etc.…”

“Because I'm Bad at the Game!” A Microanalytic Study of Self Regulated Learning in League of Legends

“…These results also have implications for the implementation of Learning Analytics Dashboards (LADs) (Bodily and Verbert, 2017 ; Bodily et al, 2018 ). While many LADs seek to motivate and aid students through comparison of their own progress against that of their peers (Fritz, 2011 ; Santos et al, 2013 ; Park and Jo, 2015 ), previous work also found that such comparison is often undesired by students or can have the opposite effect, either de-motivating them or making it harder for them to set goals and follow plans (Reimers et al, 2015 ; Aguilar, 2016 ; Rets et al, 2021 ). Similar to these LADs, League of Legends' UI elements provide players with critical information about their performance, which includes comparisons with other players, most notably post-game.…”

Learning analytics dashboard: a tool for providing actionable insights to learners

“…However, even if they can all be addressed, it is now no longer sufficient to deploy predictive models and solely display their outputs without providing the learners with explainability of how a model arrived at a given prediction. It is also becoming more apparent that learners will engage with a LAD only if they understand how displayed values are generated (Rets et al, 2021). Liu and Koedinger (2017) argue for the importance of interpretability which leads onto actionability.…”

“…Given the financial investment and the human resource effort involved in their productionization (Mahroeian et al, 2017), it is also not altogether clear what visual elements LADs should possess, that is, what type of information is effective at triggering positive behavioral adjustments in learners, or what aspects are detrimental by potentially inducing anxiety among learners. Overall, evidence about whether learning analytics and LADs improve learning in practice is scarce (Ferguson & Clow, 2017;Guzmán-Valenzuela et al, 2021;Knight et al, 2020;Rets et al, 2021;Wilson et al, 2017) and has room for further investigation.…”