Interactive Language Learning through Speech-Enabled Virtual Scenarios

Abstract

This paper describes the evaluation of an educational game designed to give learners of foreign languages the opportunity to practice their spoken language skills. Within the speech interactive Computer-Assisted Language Learning (CALL) program, scenarios are presented in which learners interact with virtual characters in the target language using speech recognition technology. Two types of interactive scenarios with virtual characters are presented as part of the game: the one-to-one scenarios which take the form of practice question and answer scenarios where the learner interacts with one virtual character and the interactive scenario which is an immersive contextualised scenario where the learner interacts with two or more virtual characters within the scene to complete a (task-based) communicative goal. The study presented here compares learners' subjective attitudes towards the different scenarios. In addition, the study investigates the performance of the speech recognition component in this game. Forty-eight students of English as a Foreign Language (EFL) took part in the evaluation. Results indicate that learners' subjective ratings for the contextualised interactive scenario are higher than for the one-to-one, practice scenarios. In addition, recognition performance was better for these interactive scenarios.

Refining pragmatically-appropriate oral communication via computer-simulated conversations

“…The agents can produce not only verbal output, but also paralinguistic cues such as frowning, smiling, hand-waving, and walking away. Due to the limitations of automatic speech recognition (ASR) technology and natural language processing (NLP) algorithms, SDS are generally built around restricted contexts or domains, such as buying a train ticket, so that possible learner input to the system is maximally predicted and speech recognition and feedback error rates are minimized (Eskenazi, 2009;Morton et al, 2012). In the reviews of language learning technologies in general (Golonka, Bowles, Frank, Richardson, & Freynik, 2014) and dialogue-based CALL applications in particular (Bibauw et al, 2015), the consensus is that SDS in its many forms, such as intelligent tutoring systems, games and virtual worlds, spoken dialogue systems, and chatbots, will likely see continuous growth in the future.…”

“…Existing SDS include DEAL (Wik & Hjalmarsson, 2009), DISCO (van Doremalen, Boves, Colpaert, Cucchiarini, & Strik, 2016), ISLAND (McGraw & Seneff, 2007), SCILL (Seneff, Wang, & Zhang, 2004), and SPELL (Morton & Jack, 2005) (see Bibauw et al, 2015, andEskenazi, 2009, for detailed descriptions of these and other programs). By focusing on SPELL, Morton et al (2012 ) illustrate how such programs generally work. For example, 'in the About Train Times scenario, the virtual character asks the learner some questions about the departure and arrival times of trains in Great Britain.…”

Effects of Educational Context on Learners’ Ratings of a Synthetic Voice

“…This type of platform is frequently referred to as a 'serious game'. Until recently there have been few instances of the integration of TTS into serious CALL games [14]- [17]. The use of TTS synthetic voices becomes essential in CALL platforms which are genuinely interactive in a non-linear fashion.…”

Development of a Mobile Application for Listening and Scaffolded Autonomous Summary Speaking and Its Effectiveness for Increasing Fluency

“…For example, HELLO, which was developed by Liu and Chu (13) , presented learners with English activities outside the classroom, such as conversation role playing with a virtual learning tutor, a story relay race, and so on. In the system of Morton et al (14) , learners could conduct a realistic interactive conversation with a virtual agent using certain scenarios. Hwang et al (15) created a game-based learning environment in which learners could tell information or news based on photos or pictures they collected.…”