No abstract

This paper explores the use of embodied conversational agents (ECAs) and their visual communicative ability to improve interaction with spoken language dialogue systems (SLDSs) through an experimental case study in the application context of secure access by speaker verification followed by remote home automation control. After identifying a set of typical interaction problems with SLDSs and associated with each of them a particular ECA gesture or behaviour, we conducted a comparative evaluation based on ITU recommendations for the evaluation of spoken dialogue systems. User tests were carried out dividing the test users into two groups, each facing a different interface setup: one with an ECA, and the other only with voice output. The ECA group encountered fewer interaction problems. Users' impressions, however, were similar in both groups, with a slight advantage observed for the ECA group. In particular, the ECA seems to help users to better understand the flow of the dialogue and reduce confusion. Results also suggest that rejection (based on privacy and security concerns) is a dimension in its own right that may influence subjective evaluation parameters closely related to user acceptance.

This chapter describes a collection of experiences and recommendations related with the design and evaluation of interactive applications integrating Embodied Conversational Agents (ECA) technology in real environments of use with children in Special Education. Benefits and challenges of using ECAs in this context are presented. These benefits and challenges have guided the creation of Special Education reinforcement applications incorporating ECAs, which have been used for extended periods of time at Infanta Elena Special Education School in Madrid. Co-design principles were applied in the development of two of the applications discussed here, with the participation of the school’s teaching staff and children with severe motor and mental disabilities (mainly with cerebral palsy). From the design experience a set of recommendations and observations were extracted, which the authors hope may serve as guidance for the scientific and educational communities when undertaking further research. For example, in an application to reinforce the learning of emotions it believe it beneficial to include ECAs that display a number of exaggerated facial expressions together with a combination of auditory and gestural reinforcements. The ECA should show its eyes and mouth clearly, in order to help the children focus their attention. These and other ECA strategies have been analysed to provide reinforcement in learning and also to attract the children’s attention when interacting with the application.

No abstract

One of the major challenges for dialogue systems deployed in commercial applications is to improve robustness when common low-level problems occur that are related with speech recognition. We first discuss this important family of interaction problems, and then we discuss the features of non-verbal, visual, communication that Embodied Conversational Agents (ECAs) bring ‘into the picture’ and which may be tapped into to improve spoken dialogue robustness and the general smoothness and efficiency of the interaction between the human and the machine. Our approach is centred around the information provided by ECAs. We deal with all stages of the conversation system development process, from scenario description, to gesture design and evaluation with comparative user tests. We conclude that ECAs can help improve the robustness of, as well as the users’ subjective experience with, a dialogue system. However, they may also make users more demanding and intensify privacy and security concerns.