Predicting Working Memory in Healthy Older Adults Using Real-Life Language and Social Context Information: A Machine Learning Approach

Abstract

Background Language use and social interactions have demonstrated a close relationship with cognitive measures. It is important to improve the understanding of language use and behavioral indicators from social context to study the early prediction of cognitive decline among healthy populations of older adults. Objective This study aimed at predicting an important cognitive ability, working memory, of 98 healthy older adults participating in a 4-day-long naturalistic observation study. We used linguistic measures, part-of-speech (POS) tags, and social context information extracted from 7450 real-life audio recordings of their everyday conversations. Methods The methods in this study comprise (1) the generation of linguistic measures, representing idea density, vocabulary richness, and grammatical complexity, as well as POS tags with natural language processing (NLP) from the transcripts of real-life conversations and (2) the training of machine learning models to predict working memory using linguistic measures, POS tags, and social context information. We measured working memory using (1) the Keep Track test, (2) the Consonant Updating test, and (3) a composite score based on the Keep Track and Consonant Updating tests. We trained machine learning models using random forest, extreme gradient boosting, and light gradient boosting machine algorithms, implementing repeated cross-validation with different numbers of folds and repeats and recursive feature elimination to avoid overfitting. Results For all three prediction routines, models comprising linguistic measures, POS tags, and social context information improved the baseline performance on the validation folds. The best model for the Keep Track prediction routine comprised linguistic measures, POS tags, and social context variables. The best models for prediction of the Consonant Updating score and the composite working memory score comprised POS tags only. Conclusions The results suggest that machine learning and NLP may support the prediction of working memory using, in particular, linguistic measures and social context information extracted from the everyday conversations of healthy older adults. Our findings may support the design of an early warning system to be used in longitudinal studies that collects cognitive ability scores and records real-life conversations unobtrusively. This system may support the timely detection of early cognitive decline. In particular, the use of a privacy-sensitive passive monitoring technology would allow for the design of a program of interventions to enable strategies and treatments to decrease or avoid early cognitive decline.

Charting everyday activities in later life: Study protocol of the mobility, activity, and social interactions study (MOASIS)

“…These data provide a rich basis to apply a wide range of emerging analytical tools from longitudinal data analyses that address the various levels of analysis, to emerging approaches of multimodal data integration and machine learning. In particular, compared to prior studies that included only a handful of days of audio recordings (Yordanova et al, 2019;Ferrario et al, 2020Ferrario et al, , 2022, our study offers sufficient amount of data that is necessary to develop machine learning approaches that can help to automate key tasks in speech data analytics (i.e., speaker identification) for naturally occurring speech in daily real-life contexts (i.e., noisy data) and would, in turn, reduce time and personnel efforts required to (pre-)process the data. Eventually, the first 30-day measurement period (as well as the accompanying baseline, intermediate and posttest assessments) will be repeated longitudinally to examine the interplay between daily lifestyle activity variability and long-term development.…”

“…The audio data assessments were modeled after the Electronically Activated Recorder (EAR; Mehl et al, 2001) and consist of 50-s ambient sound snippets sampled every 18 min that contain human speech recordings of the participant as a sensor-based indicator of social interactions in daily life. Such data can then be analyzed regarding social behavior, using coding or transcription and automatic language processing approaches (e.g., Yordanova et al, 2019;Ferrario et al, 2020Ferrario et al, , 2022. Based on linguistic characteristics of the speech utterances, inferences on cognitive activity are also possible (e.g., Luo et al, 2019Luo et al, , 2020.…”

“…Specifically, conversation content, relating to reminiscence or future- and past-time orientation, have been shown to differ by age between young and older adults ( Demiray et al, 2018 ; Haas et al, 2022 ) and by interindividual differences in well-being ( Brianza and Demiray, 2019 ; Demiray et al, 2019 ). Language features extracted from audio recordings, such as use of complex language, have also been shown to be associated with interindividual differences in chronological age ( Luo et al, 2019 , 2020 ), executive functioning ( Polsinelli et al, 2020 ), and working memory ( Ferrario et al, 2022 ). Indicators of physical activities could be diverse, including volume (e.g., time spent on walking, step count), patterns (e.g., length of walking bouts), and variability of bout lengths, which were shown to relate to older adults’ cognitive health ( Mc Ardle et al, 2020 ).…”

“…Second, closely-spaced data from sensing technology also enable research to uncover novel activity patterns in time series, such as sequential complexity ( Paraschiv-Ionescu et al, 2012 ), temporal regularity ( D’Mello and Gruber, 2021 ), and time-lagged patterns and trajectories ( Kuppens and Verduyn, 2017 ). Additionally, large volumes of daily activities data from sensing technology enable big data analytics, such as using machine learning methods in predicting older adults’ health and well-being ( Ferrario et al, 2020 , 2022 ).…”

The Robustness of Counterfactual Explanations Over Time

“…2 For these reasons, researchers in xAI are typically interested in the generation of counterfactual explanations for individuals (e.g., customers, convicts, students, and patients in digital health interventions) [10], [20], [32]- [34]. Therefore, we will typically refer to an "individual" as the data point whose machine learning outcome is explained with a counterfactual.…”