The multimodal CorpAGEst corpus: keeping an eye on pragmatic competence in later life

Abstract

The CorpAGEst project aims to study the pragmatic competence of very old people (75 years old and more), by looking at their use of verbal and gestural pragmatic markers in real-world settings (versus laboratory conditions). More precisely, we hypothesise that identifying multimodal pragmatic patterns in language use, as produced by older adults at the gesture–speech interface, helps to better characterise language variation and communication abilities in later life. The underlying assumption is that discourse markers (e.g., tu sais ‘you know’) and pragmatic gestures (e.g., an exaggerated opening of the eyes) are relevant indicators of stance in discourse. This paper's objective is mainly methodological. It aims to demonstrate how the pragmatic profile of older adults can be established by analysing audio and video data. After a brief theoretical introduction, we describe the annotation protocol that has been developed to explore issues in multimodal pragmatics and ageing. Lastly, first results from a corpus-based study are given, showing how multimodal approaches can tackle important aspects of communicative abilities, at the crossroads of language and ageing research in linguistics.

A Multimodal corpus to check on pragmatic competence for Mild Cognitive Impaired aging people

“…The functions initially identified from these works are the following four: referential, expressive, structuring and interactive, integrating the analysis model of the functional levels of language described by Halliday (1970). A theoretical combination has been established and published by Bolly & Boutet (2017) in their transversal multimodal study of healthy elderly communication, which shares the same foundations as those described earlier by Allwood (2010 -MUMIN Project).…”

“…Halliday distinguishes three functions within discourse: ideational, interpersonal and textual. The rhetorical function is one more function described by a more recent works of ; Bolly & Boutet (2017).…”

“…In doing so, we obtained a sufficient representativeness of the studied population, comparability between the subcomponents of the corpus and the proposed tasks, as well as interoperability between the tools in order to systematize the analysis. The data processing consists of 6 steps: (i) development of an interview protocols inspired both by those written in CorpAGEst (Bolly & Boutet 2017) and our clinical experience, (ii) selection of participants, collection of field data for 14 months, sampling phase, digitization of video and audio data and their editing on the ORTOLANG scientific platform, (iii) transcription and alignment of audio data, (iv) annotation of audio and video data, (v) a unimodal and multimodal analyses of data, and (vi) systematic storage of original sources and annotated files.…”

Braporus, spoken corpus of heritage russian in Brazil

“…In addition, the speech of elderly people can bring some age-specific characteristics that do not come from their bilingual conditions even when the aging is healthy (BOT; MAKONI, 2005). To characterize the age effects on the speech and to study the language variation and communication abilities in later life, corpora of speech of elderly people have recently started to be collected (BOLLY; BOUTET, 2018). The aging lexicon is affected by both environmental exposure and several cognitive mechanisms associated with learning, representation, and retrieval of information (WULLF et al, 2019).…”

Contrasting signed and spoken languages

“…It shows how investigating a shared interactive practice in a signed and a spoken language strengthens the position for such phenomena to be part of language. The comparison is based on three hours of video-recorded material drawn from three multimodal corpora of LSFB and Belgian French discourses, namely the CorpAGEst Corpus (Bolly and Boutet, 2018), the LSFB Corpus (Meurant, 2015) and the ongoing FRAPé Corpus (see Section 3.2). The studied sample gathers the productions of 12 participants, eight in French and four in LSFB, in conversational tasks.…”