Field evaluation with cognitively-impaired older adults of attention management in the Embodied Conversational Agent Louise

Abstract

We present the first experiment we conducted to evaluate the attention monitoring performance of Louise, following a Wizard of Oz method, during the interactions with a cohort of 8 elderly users in a day hospital environment. Louise is a new, semi-automatic prototype of an Embodied Conversational Agent (ECA), a virtual character interacting with users through social-like communication, adapted to the special needs of older adults with cognitive impairment; it is intended to ultimately provide assistance in their activities of daily living. We recorded and analyzed both videos of the conversation-like interactions and Louise's tracking data. In our experiment, Louise's attention estimation algorithm achieved about 80% accuracy; moreover, in almost all cases, the user's attention was successfully recaptured by Louise after a planned, experimenter-induced distraction. These results are in line with what was observed in previous experiments involving only younger adults, thus suggesting that attention measurement tools embedded in cognitive prostheses will not need to be adapted to elderly patients. Finally, to gain further insights on conversation management and provide evidence-based suggestions for future work, we performed an anthropological analysis of the whole experiment.

Usability assessment of interaction management support in LOUISE, an ECA-based user interface for elders with cognitive impairment

“…The first issue we observed is that some of the test participants tended to forget that they could only answer by "yes" or "no" and answered incorrectly, particularly the participants with the most severe memory impairment, scoring less than 20/30 at the MMSE, who account for 29 of the 56 incorrect answers (52%). This confirms the observations revealed in the anthropological analysis we conducted in the first round of experiments, reported in [41], that people with cognitive impairment tend to provide more elaborate answers and interact in a "social" way. Given this result, our error management strategy that lists the possible answers proved useful, as participants could correct their mistakes in all cases, especially after the system was changed for the valid answer reminder to be performed after two errors instead of three.…”

“…Our interaction manager could very well handle open questions with multiple possible answers, though choosing the appropriate response could be challenging and would likely require some language processing of the user's answers, to interpret his or her intents, in the behavior analysis stage. However, with our target user group, we have identified that narrow or contrasted questions, such as yes/no questions, should be privileged [41]. This is also in line with the recommendations of Zajicek [44], who suggested that menus in speech-based interfaces for older adults should be limited to 3 items at a time.…”

“…Our attention estimation method relies solely on determining, in real time, if the user is gazing towards the screen or away from it. It combines the measures of orientation of the user's shoulders, as done in [37] and [38], and head pose, as in [39], seen here as proxies for his or her intensity of attention towards LOUISE ( [40,41])…”

“…For each of the basic actions that compose the scenarios, we propose a specific conversational structure. These are based on the observations made in a previous study [41], in which we conducted an anthropological analysis of interactions between elders and a Wizard of Oz version of LOUISE. We also took inspiration from the knowledge of the symptoms of dementia.…”

Towards expressive prosody generation in TTS for reading aloud applications

“…Therefore, there is reason to assume that a conversational application considering the notions of content packaging by means of the relation between thematicity and prosody will benefit from the same advantages as in natural conversation environments. Most of all, conversational avatars in applications for children in educational settings [16], applications for those with special needs [1] as well as for elderly [2] and, in particular, for those with cognitive impairments [17], would greatly benefit from such a communicatively-oriented improvement.…”