The CABB dataset: A multimodal corpus of communicative interactions for behavioural and neural analyses

Eijk, Lotte; Rasenberg, Marlou; Arnese, Flavia; Blokpoel, Mark; Dingemanse, Mark; Doeller, Christian F.; Ernestus, Mirjam; Holler, Judith; Milivojevic, Branka; Özyürek, Aslı; Pouw, Wim; Rooij, Iris van; Schriefers, Herbert; Toni, Ivan; Trujillo, James P.; Bögels, Sara

doi:10.1016/j.neuroimage.2022.119734

Cited by 11 publications

(7 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For our current confirmatory analyses we analyze 42 pairs of participants from the CABB dataset (Eijk et al, 2022). These pairs consist of 17 all-female pairs, 5 all-male pairs, and 20 mixed-gender pairs.…”

Section: Methodsmentioning

confidence: 99%

“…The names were first corrected by removing characters (indicated here between <>, e.g. <'>, <">, <()>, <&>, <+>, <.>, <;>), converting characters <-> and </> into a space, <=> into the word <is>, correcting obvious spelling errors, changing uppercase into lowercase characters, and changing numbers into the corresponding number words (Eijk et al, 2022). Using SNAUT, the words were then checked against the corpus used for word2vec calculations (see below).…”

Section: Measures Of Communicative Effectmentioning

confidence: 99%

“…The dataset (Eijk et al, 2022) consists of 42 pairs of participants for which the individual pre and post tasks can be used, participants gave consent to share audio-video recordings of the interactions with external researchers, and transcriptions of the interaction are available (including annotations of trials etc. ), which are all necessary for our analyses.…”

Section: Supplementary Materialsmentioning

confidence: 99%

See 2 more Smart Citations

There is a power law of joint communicative effort and it reflects communicative work

Bogels,

Li,

Rasenberg

et al. 2024

Preprint

View full text Add to dashboard Cite

A drive towards efficiency seems to regulate communicative processes and ultimately language change. In line with efficiency principles, signed, spoken, and/or gestural utterances tend to reduce in overall effort over repeated referrals in referential tasks. Such reduction is often studied in individuals, using a single communicative modality. Here we seek to understand reduction of communicative effort in its natural communicative environment, i.e. during multimodal and collaborative face-to-face dialogues about displaced referents. We ascertain that the reduction in joint effort over repeated referrals actually follows a negative power relationship. This reduction in communicative effort is multimodal, occurring across gesture, speech, prosody, and turn taking, and it is interactive, based on joint effort. The effect is robust, being confirmed through a reanalysis of published datasets about (individual) effort reduction. Crucially, the effect is also communicatively relevant. The coefficient of the power relationship predicts change and convergence in interlocutors’ conceptualizations of the communicative referents. Assuming that the coefficient of the negative power relationship reflects how well effort translates into mutual understanding - a process we call communicative work - we suggest that the power function captures a complementary strategy of increasing initial exploration and applying efficient selection for an effective joint conceptualization of referents. The current report invites linguistic theory, agent-based modeling, and experimental psychological inquiries into understanding the general principles of what could amount to a ‘power law of joint communicative work’.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Measures Of Communicative Effectmentioning

confidence: 99%

Section: Supplementary Materialsmentioning

confidence: 99%

See 1 more Smart Citation

There is a power law of joint communicative effort and it reflects communicative work

Bogels,

Li,

Rasenberg

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…This could be attributed primarily to logistical difficulties in acquiring cross-cultural and crosscontextual data on children's social interactions. Indeed, face-to-face conversational data is typically acquired using two (or more) cameras positioned in such a way as to provide a frontal view of each interlocutor [e.g., 11,36]. This method provides "clean" data, but the equipment is expensive and challenging to deploy homogeneously at scale, especially across a representative range of countries where researchers do not always have access to such resources.…”

Section: ) Cultural and Individual Variabilitymentioning

confidence: 99%

Understanding Children's Multimodal Conversational Development: Challenges and Opportunities

Fourtassi¹

2023

Preprint

View full text Add to dashboard Cite

Children's ability to master skills for coordinated conversations is crucial for their healthy social and cognitive development. However, until recently, researchers lacked the tools to adequately characterize this complex phenomenon whereby children must learn to coordinate across multiple levels, use multimodal signaling, and adapt to their communicative cultural conventions and the diversity of conversational contexts. This paper proposes that the community capitalize on new technological opportunities — namely, unmoderated online data acquisition methods and high-scalability Machine Learning tools — to build a quantitative framework that will not only provide a deeper understanding of how children grow to become competent communicators but also constitute a theoretical foundation for applications in health, education, and child-oriented AI.

show abstract

“…The sixth conversation was a task-oriented dialogue (38 min in total), in which two females, who were unacquainted at the time of data collection, performed a description-drawing task with images of 2D objects (adapted from Eijk et al, 2022;. The task-oriented dialogues were collected with the objective to explore interactive alignment at the discourse level (Hu & Degand, 2022).…”

Section: Datasetmentioning

confidence: 99%

The Conversational Discourse Unit: Identification and Its Role in Conversational Turn-taking Management

Hu,

Degand

2023

dad

View full text Add to dashboard Cite

This study investigates how discourse segmentation and turn-taking interact. Mapping syntactic, prosodic and pragmatic units, five types of conversational discourse units (CDU) were identified. Based on this segmentation, associations were examined between the syntactic, prosodic and pragmatic boundaries and turn-taking, as well as the transition speed after each type of CDU. Results show: 1) The relationships between the three linguistic boundaries and the occurrence of turn-taking were significant, and the association was the strongest for the pragmatic boundaries; it was weaker for prosodic boundaries and the weakest for the syntactic boundaries. 2) The type of CDU influenced the transition speed, with the pragmatic-syntax-bound CDU being fastest. The study highlights the importance of meaning-connection and earlier emergence of the utterance gist in timing turn-taking.

show abstract

The CABB dataset: A multimodal corpus of communicative interactions for behavioural and neural analyses

Cited by 11 publications

References 61 publications

There is a power law of joint communicative effort and it reflects communicative work

There is a power law of joint communicative effort and it reflects communicative work

Understanding Children's Multimodal Conversational Development: Challenges and Opportunities

The Conversational Discourse Unit: Identification and Its Role in Conversational Turn-taking Management

Contact Info

Product

Resources

About