Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults

Abstract

The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot’s social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners’ activity and engagement through within- and between-strategy changes of the robot’s interaction with learners.

Learner and teacher perspectives on robot-led L2 conversation practice

“…The importance of the robot’s role to monitor and distribute conversation turns when one learner dominates is clearly illustrated in the Appendix. Determining the engagement level of the speaker, using textual and acoustic analysis of vocal features, and the non-active listener, using video analysis of facial expressions is valuable to decide when a topic or addressee shift is needed (Engwall, Cumbal, Lopes, Ljung & Månsson, 2022).…”

Adaptive Robot Discourse for Language Acquisition in Adulthood

“…As the quality of the interaction does not depend only on the dialogue content and strategy, it is critical to assess the affective state of the learner. Various works have highlighted the importance of tracking a learner's task and emotional engagement [14]- [16]. This distinction attempts to differentiate disinterested participants from a user that displays reduced participation from knowledge limitations.…”