Designing Hybrid Human–AI Orchestration Tools for Individual and Collaborative Activities: A Technology Probe Study

Echeverria, Vanessa; Yang, Kexin; Lawrence, LuEttaMae; Rummel, Nikol; Aleven, Vincent

doi:10.1109/tlt.2023.3248155

Cited by 8 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classroom probe (Echeverria et al, 2023) Wizard-of- worked with a total of 76 teachers. Each teacher took part in one to three methods described below; no teacher took part in all aspects of the design process.…”

Section: Methods Description Year Teachers Data Analysedmentioning

confidence: 99%

“…Orchestrating classroom learning becomes vastly more taxing when learning approaches are complex. Dynamic transitions, for instance, require teachers to manage students working on individual and collaborative learning and identify when they should switch between learning activities (Echeverria et al, 2023). Leveraging the knowledge learning instruction (KLI) framework (Koedinger et al, 2012), collaborative learning offers opportunities for mutual elaboration and coconstruction of knowledge.…”

Section: Co‐orchestrating Dynamic Learning Transitionsmentioning

confidence: 99%

See 1 more Smart Citation

How teachers conceptualise shared control with an AI co‐orchestration tool: A multiyear teacher‐centred design process

Lawrence,

Echeverria,

Yang

et al. 2023

Brit J Educational Tech

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un‐planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co‐design process comprising six design methods to understand how teachers conceptualise sharing control with an AI co‐orchestration tool, called Pair‐Up. We worked with 76 middle school math teachers, each taking part in one to three methods, to create a co‐orchestration tool that supports dynamic combinations of individual and collaborative learning using two AI‐based tutoring systems. We leveraged qualitative content analysis to examine teachers' views about sharing control with Pair‐Up, and we describe high‐level insights about the human‐AI interaction, including control, trust, responsibility, efficiency, and accuracy. Secondly, we use our results as an example showcasing how human‐centred learning analytics can be applied to the design of human‐AI technologies and share reflections for human‐AI technology designers regarding the methods that might be fruitful to elicit teacher feedback and ideas. Our findings illustrate the design of a novel co‐orchestration tool to facilitate the transitions between individual and collaborative learning and highlight considerations and reflections for designers of similar systems. Practitioner notesWhat is already known about this topic: Artificial Intelligence (AI) can help teachers facilitate complex classroom activities, such as having students move between individual and collaborative learning in unplanned ways. Designers should use human‐centred design approaches to give teachers a voice in deciding what AI might do in the classroom and if or how they want to share control with it. What this paper adds: Presents teacher views about how they want to share control with AI to support students moving between individual and collaborative learning. Describes how we adapted six design methods to design AI features. Illustrates a complete, iterative process to create human‐AI interactions to support teachers as they facilitate students moving from individual to collaborative learning. Implications for practice: We share five implications for designers that teachers highlighted as necessary when designing AI‐features, including control, trust, responsibility, efficiency and accuracy. Our work also includes a reflection on our design process and implications for future design processes.

show abstract

Section: Methods Description Year Teachers Data Analysedmentioning

confidence: 99%

Section: Co‐orchestrating Dynamic Learning Transitionsmentioning

confidence: 99%

How teachers conceptualise shared control with an AI co‐orchestration tool: A multiyear teacher‐centred design process

Lawrence,

Echeverria,

Yang

et al. 2023

Brit J Educational Tech

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to produce timely and adaptive visualisations, they are powered by pattern recognition and artificial neural networks (Gobert et al, 2023). Visualisations from AI dashboards could be used to monitor students' collaboration in online learning (Echeverria et al, 2023) and to make team members aware of their strengths and weaknesses when they plan further activities (Gobert et al, 2023;Pardo et al, 2018). Therefore, AI-enhanced dashboards could play a useful role in the development of collaboration skills.…”

Section: Ai and Its Affordances For Supporting 21st-century Skillsmentioning

confidence: 99%

The affordances of artificial intelligence-based tools for supporting 21st-century skills:

Celik,

Gedrimiene,

Siklander

et al. 2024

AJET

View full text Add to dashboard Cite

Twenty-first-century skills should be integrated into higher education to prepare students for complex working-life challenges. Artificial intelligence (AI)-powered tools have the potential to optimise skill development among higher education students. Therefore, it is important to conceptualise relevant affordances of AI systems for 21st-century skills development in higher education. This study aimed to present an overview of journal articles published in the Web of Science database that specifically addressed the affordances of AI-based tools for 21st-century skills development. Four distinct categories of AI-based tools (intelligent tutoring systems, chatbots, AI-powered dashboards and automated grading systems) were identified as capable of promoting six main 21st-century skills (collaboration, communication, creativity, critical thinking, information and communication technology and problem-solving). The review revealed that the utilisation of AI-based tools might contribute to the simultaneous development of multiple 21st-century skills (e.g., collaboration and critical thinking). The results showed that adaptive feedback from AI plays a significant role as a facilitator in the development of 21st-century skills. Furthermore, the utilisation of diverse functional AI affordances (e.g., prediction and profiling) might contribute to the development of various skills. AI-based technologies appeared to target the 21st-century skills of problem-solving and its subskills the most. Implications for practice or policy: More functional affordances of AI (e.g., prediction and profiling) should be employed in AI-based tools. This could support higher education students’ 21st-century skills. AI-based tools (e.g., chatbots and intelligent tutors) interact with end users through their data. AI systems have the potential to promote 21st-century skills by using students’ multimodal data. AI technologies should be more integrated into the social sciences and humanities in the higher education context to support students’ 21st-century skills.

show abstract

“…Typically, researchers and developers design an augmented interface and connect it with tangible hardware objects that children interact with, either through touch or keyboard devices [23]. For instance, a study by [24] developed a number system incorporating collaborative learning features with screen-based multitouch devices. Multitouch and multikeyboard technologies have become prevalent, allowing learners to connect with the digital learning context.…”

Section: Introductionmentioning

confidence: 99%

TARES: A Game-Based Tangible Augmented Reality English Spelling Mastery System With Minimal Cognitive Load

Akoth Ongoro,

FanJiang,

Hung

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Over an extended period, spelling lessons have consistently played a crucial role in language acquisition, commonly integrated into educational settings to enhance learners' spelling proficiency. Traditional classroom environments often require learners to spell and write words, leading those facing persistent difficulties to experience anxiety and social disconnection. This challenge is particularly notable in preschool settings where a more interactive approach to spelling sessions could benefit children with difficulties. This research develops a screenless language learning system for spelling acquisition, with a specific focus on minimizing digital screen exposure for preschoolers. The proposed virtual augmented interface connects with tangible elements and game features. The tangible interface becomes the primary tool for learning, and the system achieves complete accuracy without necessitating learners to hold additional digital devices. The proposed system integrates augmented reality, tangible user interfaces, and serious game-based learning. Utilizing 3D Unity and Vuforia technology, augmented reality and serious game components are employed to create spelling content. The study highlights the importance of tangible letters and objects in conjunction with real-world items to facilitate cooperative and collaborative learning, encouraging social interaction among learners. The proposed learning content levels are adjusted to the age of learners (4-6 years) for alleviating cognitive overload. For gamification, the system incorporates three difficulty levels tailored to three, four, and five-letter words. This structured approach systematically enhances spelling skills while simultaneously addressing the challenge of excessive screen device exposure.

show abstract

Designing Hybrid Human–AI Orchestration Tools for Individual and Collaborative Activities: A Technology Probe Study

Cited by 8 publications

References 42 publications

How teachers conceptualise shared control with an AI co‐orchestration tool: A multiyear teacher‐centred design process

How teachers conceptualise shared control with an AI co‐orchestration tool: A multiyear teacher‐centred design process

The affordances of artificial intelligence-based tools for supporting 21st-century skills:

TARES: A Game-Based Tangible Augmented Reality English Spelling Mastery System With Minimal Cognitive Load

Contact Info

Product

Resources

About