On the multimodality of [all the way from X PREP Y]

Abstract

This paper presents a case study on the English construction [all the way from X PREP Y] and its co-occurrence with manual gestures in multimodal television data from the Red Hen database.

Indicating verbs as typologically unique constructions: Reconsidering verb ‘agreement’ in sign languages

“…The particular patterns with sets of verbs, or even individual verbs, has yet to be explored in any detail for BSL, but it is likely that the idiosyncratic formational features and semantics of particular indicating verb constructions are interacting with their contexts of use here. This is similar to idiosyncrasies found in the various gestural patterns identified in Zima's (2017a; work. Indeed, the idiosyncrasy in the indicating verb system is such that learners need to know which signs act as indicating verbs and which do not, which forms are single, double or backwards indicating verbs, which specific verbs often show directionality and which exhibit this pattern less often, and how these variable features interact with other aspects of the language.…”

“…Thus, Liddell argues, any movement of the hand towards such a location signals an association with the referent in the same way as a pointing gesture would by a non-signer. We extend this analysis here by proposing, building on work by gesture researchers (e.g., Andrén 2010;Harrison 2010;Zima 2017a;, that these verbs are typologically unique unimodal constructions (comparable to multimodal constructions in spoken languages).…”

“…agreement analysis has grown, though many alternative accounts (e.g., Wilbur 2013;Wilcox & Occhino 2016) do not accept the pointing gesture analysis in Liddell's proposal. In this paper, we build on Liddell's (2003) cognitive grammar analysis of indicating verbs by introducing to his account some key concepts from Construction Grammar, and by specifically suggesting that these represent a sign language equivalent to 'multimodal constructions' found in spoken languages (as proposed by Andrén 2010;Harrison 2010;Zima 2017a;Blackwell et al 2015 amongst others). We also discuss some issues with the agreement analysis, demonstrate how these can be accounted for under the novel Construction Grammar approach we outline here, and explore how supporting evidence is emerging from corpus-based studies of indicating verbs.…”

“…Just like unimodal spoken language constructions, he suggests that multimodal constructions range from relatively fixed (e.g., "word like") constructions to more flexible and productive combinations (e.g., "grammar like"). More recently, Zima (2017a; investigates the use of the semi-lexicalised English constructions [V(motion) in circles] and [all the way from X PREP Y]. The data were collected from an audio-visual corpus of spontaneous language samples from a range of discourse types.…”

When Gesture “Takes Over”: Speech-Embedded Nonverbal Depictions in Multimodal Interaction

“…In addition, the unbalanced interpersonal power dynamics found in certain settings (e.g., in contexts of instruction, see Goldin-Meadow, 2003 ; Hsu et al, to appear ) are to a large extent absent. Indeed, a growing number of studies on multimodal communication have examined television data for similar reasons [see the studies drawing on the databases of the Red Hen Lab ( www.redhenlab.org ) and the TV News Archive ( archive.org/details/tv ); e.g., Steen and Turner, 2013 ; Winter et al, 2013 ; Zima, 2017 ; Hinnell, 2018 , 2019 ]. On top of all these, TV talk shows abound in recounts of past experiences and enactments of hypothetical scenarios, both of which contribute to the richness of depicting.…”