Smart conversational agents for reminiscence

Nikitina, Svetlana; Callaioli, Sara; Báez, Marcos

doi:10.1145/3195555.3195567

Cited by 30 publications

(13 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous research focused on reminiscence as a therapy against dementia have shown the efficacy of information and communication technologies [ 14 ]; in particular, studies addressing the topic of automated reminiscence therapy aim at eliciting reminiscence from the users rather than detecting the presence of reminiscence in everyday life [ 17 , 18 ]. These studies have assumed that reminiscence is elicited in a setting with a human companion (eg, a therapist) or through a digital companion device using the Wizard of Oz technique [ 19 ].…”

Section: Discussionmentioning

confidence: 99%

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

Ferrario¹,

Demiray²,

Yordanova³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

Background Reminiscence is the act of thinking or talking about personal experiences that occurred in the past. It is a central task of old age that is essential for healthy aging, and it serves multiple functions, such as decision-making and introspection, transmitting life lessons, and bonding with others. The study of social reminiscence behavior in everyday life can be used to generate data and detect reminiscence from general conversations. Objective The aims of this original paper are to (1) preprocess coded transcripts of conversations in German of older adults with natural language processing (NLP), and (2) implement and evaluate learning strategies using different NLP features and machine learning algorithms to detect reminiscence in a corpus of transcripts. Methods The methods in this study comprise (1) collecting and coding of transcripts of older adults’ conversations in German, (2) preprocessing transcripts to generate NLP features (bag-of-words models, part-of-speech tags, pretrained German word embeddings), and (3) training machine learning models to detect reminiscence using random forests, support vector machines, and adaptive and extreme gradient boosting algorithms. The data set comprises 2214 transcripts, including 109 transcripts with reminiscence. Due to class imbalance in the data, we introduced three learning strategies: (1) class-weighted learning, (2) a meta-classifier consisting of a voting ensemble, and (3) data augmentation with the Synthetic Minority Oversampling Technique (SMOTE) algorithm. For each learning strategy, we performed cross-validation on a random sample of the training data set of transcripts. We computed the area under the curve (AUC), the average precision (AP), precision, recall, as well as F1 score and specificity measures on the test data, for all combinations of NLP features, algorithms, and learning strategies. Results Class-weighted support vector machines on bag-of-words features outperformed all other classifiers (AUC=0.91, AP=0.56, precision=0.5, recall=0.45, F1=0.48, specificity=0.98), followed by support vector machines on SMOTE-augmented data and word embeddings features (AUC=0.89, AP=0.54, precision=0.35, recall=0.59, F1=0.44, specificity=0.94). For the meta-classifier strategy, adaptive and extreme gradient boosting algorithms trained on word embeddings and bag-of-words outperformed all other classifiers and NLP features; however, the performance of the meta-classifier learning strategy was lower compared to other strategies, with highly imbalanced precision-recall trade-offs. Conclusions This study provides evidence of the applicability of NLP and machine learning pipelines for the automated detection of reminiscence in older adults’ everyday conversations in German. The methods and findings of this study could be relevant for designing unobtrusive computer systems for the real-time detection of social reminiscence in the everyday life of older adults and classifying their functions. With further improvements, these systems could be deployed in health interventions aimed at improving older adults’ well-being by promoting self-reflection and suggesting coping strategies to be used in the case of dysfunctional reminiscence cases, which can undermine physical and mental health.

show abstract

Section: Discussionmentioning

confidence: 99%

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

Ferrario¹,

Demiray²,

Yordanova³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…They include: Siri (Apple), Alexa (Amazon) and Google Assistant. Conversational interfaces have been studied in a variety of contexts, ranging from: health care [27], aging [31], physical activity [49,8], and for people with an intellectual disability [6].…”

Section: Related Workmentioning

confidence: 99%

"Hey Model!" Natural User Interactions and Agency in Accessible Interactive 3D Models

Reinders

Butler

Marriott

2020

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

While developments in 3D printing have opened up opportunities for improved access to graphical information for people who are blind or have low vision (BLV), they can provide only limited detailed and contextual information. Interactive 3D printed models (I3Ms) that provide audio labels and/or a conversational agent interface potentially overcome this limitation. We conducted a Wizard-of-Oz exploratory study to uncover the multi-modal interaction techniques that BLV people would like to use when exploring I3Ms, and investigated their attitudes towards different levels of model agency. These findings informed the creation of an I3M prototype of the solar system. A second user study with this model revealed a hierarchy of interaction, with BLV users preferring tactile exploration, followed by touch gestures to trigger audio labels, and then natural language to fill in knowledge gaps and confirm understanding.

show abstract

“…In line with this approach, PAs have been used to significantly improve episodic memory by using a reminiscing agent to debrief the student following exploration of a virtual world (Nicholas, Van Bergen, & Richards, 2015). Similar to the use of IVAs to assist the elderly to reminisce (Nikitina, Callaioli, & Baez, 2018), PAs could help students with remembering and act as a form of external memory aid that does more than store or retrieve information. Another direction to encourage reflection is the use of teachable agents where the human (co)-learner teaches the agent what to do or provides the agent with answers to its questions (Pareto, 2014).…”

Section: Pedagogy: Vision and New Possibilitiesmentioning

confidence: 99%

Supporting and challenging learners through pedagogical agents: Addressing ethical issues through designing for values

Richards¹,

Dignum²

2019

Brit J Educational Tech

View full text Add to dashboard Cite

Pedagogical Agents (PAs) that would guide interactions in intelligent learning environments were envisioned two decades ago. These early animated characters had been shown to deliver learning benefits. However, little was understood regarding what aspects were beneficial for learning and what sort of learning PAs were suitable for. This article considers the current and future use of PAs to support and challenge learners from three perspectives. Firstly, we look at PAs from a practical perspective to consider what Intelligent Virtual Agents are, the roles they play in education and beyond and the underlying technologies and theories driving them. Next we take a pedagogical perspective to consider the vision, pedagogical approaches supported and new possible uses of PAs. This leads us to the political perspective to consider the values, ethics and societal impacts of PAs. Drawing all three perspectives together we present a design for values approach to designing ethical and socially responsible PAs.

show abstract

Smart conversational agents for reminiscence

Cited by 30 publications

References 33 publications

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

"Hey Model!" Natural User Interactions and Agency in Accessible Interactive 3D Models

Supporting and challenging learners through pedagogical agents: Addressing ethical issues through designing for values

Contact Info

Product

Resources

About

Smart conversational agents for reminiscence

Cited by 30 publications

References 33 publications

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

Social Reminiscence in Older Adults’ Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning

"Hey Model!"  Natural User Interactions and Agency in Accessible Interactive 3D Models

Supporting and challenging learners through pedagogical agents: Addressing ethical issues through designing for values

Contact Info

Product

Resources

About

"Hey Model!" Natural User Interactions and Agency in Accessible Interactive 3D Models