More Than Figures on Your Laptop: (Dis)trustful Implementation of Learning Analytics

Tsai, Yi‐Shan; Whitelock‐Wainwright, Alexander; Gašević, Dragan

doi:10.18608/jla.2021.7379

Cited by 20 publications

(27 citation statements)

References 72 publications

(147 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This could indicate that the HEIs that were in the early stages of adoption were paying particular attention to smoothing out ethics and privacy issues or that they had concerns related to setting protocols for data sharing with external partners. An increasing number of studies have found that there are concerns raised in HEIs about sharing of student data with external stakeholders (Selwyn, 2019 ; Tsai et al, 2021b ). This concern should be addressed by setting a policy that safeguards stakeholders’ data from harmful use that can result from data sharing, especially in the preparation stage, as suggested above.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Untangling connections between challenges in the adoption of learning analytics in higher education

et al. 2022

View full text Add to dashboard Cite

Potential benefits of learning analytics (LA) for improving students’ performance, predicting students’ success, and enhancing teaching and learning practice have increasingly been recognized in higher education. However, the adoption of LA in higher education institutions (HEIs) to date remains sporadic and predominantly small in scale due to several socio-technical challenges. To better understand why HEIs struggle to scale LA adoption, it is needed to untangle adoption challenges and their related factors. This paper presents the findings of a study that sought to investigate the associations of adoption factors with challenges HEIs face in the adoption of LA and how these associations are compared among HEIs at different scopes of adoption. The study was based on a series of semi-structured interviews with senior managers in HEIs. The interview data were thematically analysed to identify the main challenges in LA adoption. The connections between challenges and other factors related to LA adoption were analysed using epistemic network analysis (ENA). From senior managers’ viewpoints, ethical issues of informed consent and resistance culture had the strongest links with challenges of learning analytic adoption in HEI; this was especially true for those institutions that had not adopted LA or who were in the initial phase of adoption (i.e., preparing for or partially implementing LA). By contrast, among HEIs that had fully adopted LA, the main challenges were found to be associated with centralized leadership, gaps in the analytic capabilities, external stakeholders, and evaluations of technology. Based on the results, we discuss implications for LA strategy that can be useful for institutions at various stages of LA adoption, from early stages of interest to the full adoption phase.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Typical stakeholders involved in LA in HEIs include students, teaching staff, institutional management, researchers, and developers (Drachsler & Greller, 2012 ; Khalil & Ebner, 2015b ; Tsai et al, 2018 , 2021b ). Stakeholders can be divided into clients and subjects (Drachsler & Greller, 2012 ; Kollom et al, 2021 ).…”

Section: Literature Reviewmentioning

confidence: 99%

Untangling connections between challenges in the adoption of learning analytics in higher education

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Moreover, LA practice and research increasingly stress LA's impact on learning (eg, Chen et al, 2018; Lang et al, 2022) and learning progression (eg, Gašević et al, 2022). Furthermore, although acceptability does not stand out as a term related to LA, it is reflected in the essential topics of trustworthiness (eg, Kitto & Knight, 2019; Tsai et al, 2021) and fairness (eg, Carter & Egliston, 2021; Coghlan et al, 2021; O'Neil, 2016; Selwyn, 2020) of LA. Finally, the aspect of cost is related, but not limited to sustainability of LA (Gašević et al, 2022).…”

Section: Theoretical Backgroundmentioning

confidence: 99%

Assessment validity and learning analytics as prerequisites for ensuring student‐centred learning design

Divjak

Svetec

Horvat

et al. 2022

Brit J Educational Tech

View full text Add to dashboard Cite

To ensure the validity of an assessment programme, it is essential to align it with the intended learning outcomes (LO). We present a model for ensuring assessment validity which supports this constructive alignment and uses learning analytics (LA). The model is based on LA that include a comparison between ideal LO weights (expressing the prioritization of LOs), actual assessment weights (maximum assessment points per LO), and student assessment results (actually obtained assessment points per LO), as well as clustering and trace data analysis. These analytics are part of a continuous improvement cycle, including strategic planning and learning design (LD) supported by LO prioritization, and monitoring and evaluation supported by LA. To illustrate and test the model, we conducted a study on the example of a graduate‐level higher education course in applied mathematics, by analysing student assessment results and activity in a learning management system. The study showed that the analyses provided valuable insights with practical implications for the development of sound LD, tailored educational interventions, databases of assessment tasks, recommendation systems, and self‐regulated learning. Future research should investigate the possibilities for automation of such LA, to enable full exploitation of their potential and use in everyday teaching and learning. Practitioner notesWhat is already known about this topic To develop sound, student‐centred learning design (LD), it is essential to ensure that assessment is constructively aligned with the intended learning outcomes (LO). This constructive alignment is crucial for ensuring the validity of an assessment program. Learning analytics (LA) can provide insights that help develop valid assessment programs. What this paper adds As not all LOs are equally important, assessment programs should reflect the prioritization of LOs, which can be determined by using various multi‐criteria decision‐making (MCDM) methods. This article presents and illustrates, based on an empirical case, a model of continuous improvement of LD, which uses LA to compare how LOs are reflected in (actual) students' results, in an (actual) assessment program, and in the (ideal) prioritization of LOs based on MCDM. The study presents how clustering of students based on their assessment results can be used in LA to provide insights for educational interventions better targeted to students' needs. Implications for practice and/or policy The proposed LA can provide important insights for the development (or improvement) of LD in line with the intended course LOs, but also study program LOs (if course and study program LOs are properly aligned). The LA can also contribute to the development of databases of assessment tasks aligned with course LOs, with ensured validity, supporting sharing and reusing, as well as to the development of tailored educational interventions (eg, based on clustering). The proposed LA can also contribute to the development of recommendation systems, with recommendations for the improvement of LD for teachers or learning suggestions for students, as well as students' meta‐cognition and self‐regulated learning.

show abstract

“…Therefore, key stakeholders should be taken as an integral component of any learning analytics implementation (Tsai et al, 2018). This is most important considering the distrust teachers may develop towards data-driven implementations, attributed mostly to the subjective nature of numbers, the fear of power diminution, and approaches to design and implementation (Tsai et al, 2021); it was recently pointed out that such a tendency is particularly prominent when referring to preditions for students at-risk (Kollom et al, 2021). On the other hand, involving teachers in the early stages of learning analytics design, particularly engaging them with data collected on students, may improve their pedagogic and assessment practises, leading potentially to the development of more suitable learning analytics tools (Holstein et al, 2019;Vezzoli et al, 2020).…”

Section: Education Stakeholders' Engagement With Prediction Modelsmentioning

confidence: 99%

Insights of Instructors and Advisors into an Early Prediction Model for Non-Thriving Students

Hershkovitz

Ambrose

2022

Learning Analytics

View full text Add to dashboard Cite

In this qualitative study (N=6), we explored insights of first-year students’ instructors and advisors into an early identification system aimed at detecting non-thriving students in the context of an all-campus first-year orientation course for undergraduates. Following the development of that prediction model in a bottom-up manner, using a plethora of available data, we focus on how its end-users could help us understand the underlying mechanisms that drive the identification of non-thriving students. As findings suggest, participants were appreciative overall of the prediction and its timing and came up with various behaviours that could explain non-thriving, mostly motivation and engagement. They suggested additional data that could predict non-thriving, including background information, academic engagement, and learning habits.

show abstract

More Than Figures on Your Laptop: (Dis)trustful Implementation of Learning Analytics

Cited by 20 publications

References 72 publications

Untangling connections between challenges in the adoption of learning analytics in higher education

Untangling connections between challenges in the adoption of learning analytics in higher education

Assessment validity and learning analytics as prerequisites for ensuring student‐centred learning design

Insights of Instructors and Advisors into an Early Prediction Model for Non-Thriving Students

Contact Info

Product

Resources

About