Orchestration in learning technology research: evaluation of a conceptual framework

Abstract

The term ‘orchestrating learning’ is being used increasingly often, referring to the coordination activities performed while applying learning technologies to authentic settings. However, there is little consensus about how this notion should be conceptualised, and what aspects it entails. In this paper, a conceptual framework for orchestration-related research is evaluated by an international panel of learning technology experts. The results of this evaluation show that the framework is complete and understandable, and it is particularly useful as an integrative list of aspects to consider when designing and evaluating learning technologies. To illustrate a way in which the framework can be used to help researchers structure their classroom innovation evaluations, an example is presented that follows the adoption of the framework by a group of researchers in Singapore. Finally, a new evolved version of the framework is presented, taking into account the evaluation feedback.Keywords: orchestration; conceptual framework; analytical lens; expert panel(Published: 24 September 2015)Citation: Research in Learning Technology 2015, 23: 28019 - http://dx.doi.org/10.3402/rlt.v23.28019

Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom

“…as crucial for effective adoption of any classroom technology Nussbaum & Diaz, 2013;Terzis & Economides, 2011). As already recognized above, innovations that are adapted to teacher-specific challenges are more likely to succeed (Prieto et al, 2015(Prieto et al, , 2011Terzis & Economides, 2011), while other authors also consider the users' perception of the technology as critical for adoption (Terzis & Economides, 2011).…”

“…(Dillenbourg, 2013;Prieto, Dimitriadis, Asensio-Pérez, & Looi, 2015;Prieto, Holenko-Dlab, Abdulwahed, Gutiérrez, & Balid, 2011). They also highlight other means of support and facilitation (e.g., lesson plans, guidelines of technology usage, etc.)…”

“…Another issue where orchestration-related literature can create great added value is in how it provides tools to characterize orchestration, both before and after the introduction of an innovation: defining the activities and tasks that classroom orchestration entails (Dillenbourg, 2013;Phiri et al, 2016;Prieto et al, 2015Prieto et al, , 2011Tchounikine, 2013), or the importance of actors and roles (i.e., who performs each of these tasks), and how the innovation changes them (Prieto et al, 2015(Prieto et al, , 2011. In this regard, some of the works on orchestration have also provided pedagogical guidelines that may support the adoption of a technology for a particular pedagogical purpose Hämäläinen & Vähäsantanen, 2011;Nussbaum & Diaz, 2013).…”

The orchestration of a collaborative information seeking learning task

“…115-116) Thus far CIS tools have largely been described in terms of their awareness and communication features, with little discussion given to the wider context of orchestration. As Prieto et al, (2015) highlight, consideration of these issues is crucial for developing an understanding of how learning technologies might be successfully adopted by individual learners and institutions. Yet this context is crucial for the success of understanding a system's role in 'searching to learn', beyond implementation within the scope of any particular experimental study.…”

“…In characterising 'orchestration', Prieto et al, (2015) thus build on Prieto et al,'s (2011) review of the literature and conceptual framework in which eight aspects were identified: five characterizing orchestration; and three characterizing key contextual factors in orchestration. As Prieto et al (2015) note, the five characterizing aspects are: (Prieto et al 2015, pp.…”