Auditory visual prominence

Moubayed, Samer Al; Beskow, Jonas; Granström, Björn

doi:10.1007/s12193-010-0054-0

Cited by 20 publications

(10 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research on co-verbal gestures has to date focused on adult speech perception, and particularly on the link between co-verbal gestures and prosodic prominence. Both head and eyebrow movements have been shown to enhance speech intelligibility in adults [15,17], as well as adults’ perception of prosodic prominence and focus (i.e., the new or contrastive information within an utterance), and hinder perception of prominence when visual and auditory prominence are incongruent [18]. Head and eyebrow movements can even change the realization of acoustic prominence [19–22].…”

Section: Introductionmentioning

confidence: 99%

Finding phrases: On the role of co-verbal facial information in learning word order in infancy

Cruz-Pavía¹,

Gervain²,

Vatikiotis‐Bateson³

et al. 2019

PLoS ONE

View full text Add to dashboard Cite

The input contains perceptually available cues, which might allow young infants to discover abstract properties of the target language. Thus, word frequency and prosodic prominence correlate systematically with basic word order in natural languages. Prelexical infants are sensitive to these frequency-based and prosodic cues, and use them to parse new input into phrases that follow the order characteristic of their native languages. Importantly, young infants readily integrate auditory and visual facial information while processing language. Here, we ask whether co-verbal visual information provided by talking faces also helps prelexical infants learn the word order of their native language in addition to word frequency and prosodic prominence. We created two structurally ambiguous artificial languages containing head nods produced by an animated avatar, aligned or misaligned with the frequency-based and prosodic information. During 4 minutes, two groups of 4- and 8-month-old infants were familiarized with the artificial language containing aligned auditory and visual cues, while two further groups were exposed to the misaligned language. Using a modified Headturn Preference Procedure, we tested infants’ preference for test items exhibiting the word order of the native language, French, vs. the opposite word order. At 4 months, infants had no preference, suggesting that 4-month-olds were not able to integrate the three available cues, or had not yet built a representation of word order. By contrast, 8-month-olds showed no preference when auditory and visual cues were aligned and a preference for the native word order when visual cues were misaligned. These results imply that infants at this age start to integrate the co-verbal visual and auditory cues.

show abstract

Section: Introductionmentioning

confidence: 99%

Finding phrases: On the role of co-verbal facial information in learning word order in infancy

Cruz-Pavía¹,

Gervain²,

Vatikiotis‐Bateson³

et al. 2019

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Moreover, from a linguistic point of view, beat gestures have been shown to serve a focus-marking function (Yasinnik, Renwick & Shattuck-Hufnagel, 2004;Jannedy & Mendoza-Denton, 2005;Loehr, 2012;Shattuck-Hufnagel, et al, 2016). In addition, adult listeners have shown an increase in prominence perception when words are produced together with hand gestures (Krahmer & Swerts, 2007) and head/facial beat gestures (Moubayed, Beskow & Granström, 2010). Apart from the abovementioned physiological and linguistic evidence, the positive cognitive effects of beat gestures are consistent with the embodied cognition paradigm, which underlines the relevance of the body movements and multimodal supporting channels in cognition and in favoring memory traces (see Paivio, 1990;andBarsalou et al, 2003, Bersalou, 2008).…”

Section: Introductionmentioning

confidence: 99%

Beat gestures improve word recall in 3- to 5-year-old children

Igualada

Esteve-Gibert

Prieto

2017

Journal of Experimental Child Psychology

View full text Add to dashboard Cite

Although research has shown that adults can benefit from the presence of beat gestures in word recall tasks, studies have failed to conclusively generalize these findings to preschool children. This study investigated whether the presence of beat gestures helps children to recall information when these gestures have the function of singling out a linguistic element in its discourse context. A total of 106 3- to 5-year-old children were asked to recall a list of words within a pragmatically child-relevant context (i.e., a storytelling activity) in which the target word was or was not accompanied by a beat gesture. Results showed that children recalled the target word significantly better when it was accompanied by a beat gesture than when it was not, indicating a local recall effect. Moreover, the recall of adjacent non-target words did not differ depending on the condition, revealing that beat gestures seem to have a strictly local highlighting function (i.e., no global recall effect). These results demonstrate that preschoolers benefit from the pragmatic contribution offered by beat gestures when they function as multimodal markers of prominence.

show abstract

“…Nonetheless, studies of interpersonal alignment at more than one analytic level consistently show close correspondences between speech, facial gesture and other body movement during conversation [ 38 , 42 , 43 ] especially in affiliative rather than argumentative contexts [ 44 ]. Importantly, alignment of visual with acoustic cues significantly benefits speech intelligibility, especially in adverse listening conditions [ 45 ]. D’Ausilio and collaborators’ [ 46 ] review evidence for similar alignment patterns in a variety of scripted ensemble musical performances.…”

Section: Introductionmentioning

confidence: 99%

Musical improvisation enhances interpersonal coordination in subsequent conversation: Motor and speech evidence

et al. 2021

View full text Add to dashboard Cite

This study explored the effects of musical improvisation between dyads of same-sex strangers on subsequent behavioural alignment. Participants–all non-musicians–conversed before and after either improvising music together (Musical Improvisation—MI—group) or doing a motoric non-rhythmic cooperative task (building a tower together using wooden blocks; the Hands-Busy—HB—group). Conversations were free, but initially guided by an adaptation of the Fast Friends Questionnaire for inducing talk among students who are strangers and meeting for the first time. Throughout, participants’ motion was recorded with an optical motion-capture system (Mocap) and analysed in terms of speed cross-correlations. Their conversations were also recorded on separate channels using headset microphones and were analysed in terms of the periodicity displayed by rhythmic peaks in the turn transitions across question and answer pairs (Q+A pairs). Compared with their first conversations, the MI group in the second conversations showed: (a) a very rapid, partially simultaneous anatomical coordination between 0 and 0.4 s; (b) delayed mirror motoric coordination between 0.8 and 1.5 s; and (c) a higher proportion of Periodic Q+A pairs. In contrast, the HB group’s motoric coordination changed slightly in timing but not in degree of coordination between the first and second conversations, and there was no significant change in the proportion of periodic Q+A pairs they produced. These results show a convergent effect of prior musical interaction on joint body movement and use of shared periodicity across speech turn-transitions in conversations, suggesting that interaction in music and speech may be mediated by common processes.

show abstract

Auditory visual prominence

Cited by 20 publications

References 27 publications

Finding phrases: On the role of co-verbal facial information in learning word order in infancy

Finding phrases: On the role of co-verbal facial information in learning word order in infancy

Beat gestures improve word recall in 3- to 5-year-old children

Musical improvisation enhances interpersonal coordination in subsequent conversation: Motor and speech evidence

Contact Info

Product

Resources

About