Charting the Design and Analytics Agenda of Learnersourcing Systems

Abstract

Learnersourcing is emerging as a viable learner-centred and pedagogically justified approach for harnessing the creativity and evaluation power of learners as experts-in-training. Despite the increasing adoption of learnersourcing in higher education, understanding students' behaviour while engaged in learnersourcing and best practices for the design and development of learnersourcing systems are still largely under-researched. This paper offers data-driven reflections and lessons learned from the development and deployment of a learnersourcing adaptive educational system called RiPPLE, which to date, has been used in more than 50-course offerings with over 12,000 students. Our reflections are categorised into examples and best practices on (1) assessing the quality of students' contributions using accurate, explainable and fair approaches to data analysis, (2) incentivising students to develop high-quality contributions and (3) empowering instructors with actionable and explainable insights to guide student learning. We discuss the implications of these findings and how they may contribute to the growing literature on the development of effective learnersourcing systems and more broadly technological educational solutions that support learner-centred learning at scale. CCS CONCEPTS• Applied computing → Computer-assisted instruction; Interactive learning environments; • Human-centered computing → Collaborative and social computing systems and tools.

Repositioning students as co-creators of curriculum for online learning resources

“…Previous research has identified many potential benefits to co-creation of curriculum between students and academic staff, including reduced power disparities, deeper engagement, deep learning, increased motivation to study, better academic literacy, and more developed peer communities (Bergmark & Westman, 2016;Cook-Sather, 2014;Deeley & Bovill, 2017;Khosravi et al, 2021;Rust et al, 2003;Shibani et al, 2020). In this study, we investigated the interaction between types of student engagement with a platform for co-creation of learning resources, and student satisfaction and academic performance.…”

Information Resilience: the nexus of responsible and agile approaches to information use

“…RiPPLE [85] is an AES that takes the crowdsourcing approach of partnering with students, also referred to as learnersourcing [84], to create the resource repository. Figure 2 uses screenshots of the platform to demonstrate some of its main functionality.…”

“…Below we outline our key learnings and insights from our experience, which spans development and adoption of RiPPLE in over 100 courses across a range of disciplines including Medicine, Pharmacy, Psychology, Education, Business, Computer Science, and Biosciences with roughly 25,000 students who have authored over 50,000 learning resources and over 2 million interactions on these resources. A more in-depth analysis of the case study is available in [84].…”

“…When flagging a resource for spot-checking, RiPPLE uses absolute and relative points of comparison to help instructors make sense of the recommendation. Figure 3 demonstrates an example of how a resource being flagged due to moderator disagreement and providing the relative points of comparison has helped to significantly improve the efficiency of expert review process [39,84].…”

Bridging the Gap Between Theory and Empirical Research in Evaluative Judgment

“…Figure 3(b) presents the RiPPLE interface for creating a multi-answer question. Not only does this reduce the cost of content generation, but it also holds the potential to foster students' higher-order skills, such as evaluative judgment (Doroudi et al, 2018;Khosravi, Demartini, Sadiq, & Gasevic, 2021). However, since students are still developing their expertise, and even though they are encouraged to reflect on the quality of their work, it is likely that some learning resources created will be ineffective, inappropriate, or incorrect (Bates, Galloway, Riise, & Homer, 2014).…”