Systematic mappings between semantic categories and types of iconic representations in the manual modality: A normed database of silent gesture

Abstract

An unprecedented number of empirical studies have shown that iconic gestures-those that mimic the sensorimotor attributes of a referent-contribute significantly to language acquisition, perception, and processing. However, there has been a lack of normed studies describing generalizable principles in gesture production and in comprehension of the mappings of different types of iconic strategies (i.e., modes of representation; Müller, 2013). In Study 1 we elicited silent gestures in order to explore the implementation of different types of iconic representation (i.e., acting, representing, drawing, and personification) to express concepts across five semantic domains. In Study 2 we investigated the degree of meaning transparency (i.e., iconicity ratings) of the gestures elicited in Study 1. We found systematicity in the gestural forms of 109 concepts across all participants, with different types of iconicity aligning with specific semantic domains: Acting was favored for actions and manipulable objects, drawing for nonmanipulable objects, and personification for animate entities. Interpretation of gesture-meaning transparency was modulated by the interaction between mode of representation and semantic domain, with some couplings being more transparent than others: Acting yielded higher ratings for actions, representing for object-related concepts, personification for animate entities, and drawing for nonmanipulable entities. This study provides mapping principles that may extend to all forms of manual communication (gesture and sign). This database includes a list of the most systematic silent gestures in the group of participants, a notation of the form of each gesture based on four features (hand configuration, orientation, placement, and movement), each gesture's mode of representation, iconicity ratings, and professionally filmed videos that can be used for experimental and clinical endeavors.

Hearing non-signers use their gestures to predict iconic form-meaning mappings at first exposure to signs

“…There is a generalised assumption that the iconic strategies to represent a referent in gesture are idiosyncratic and thus variable. However, the results of the present study support recent evidence that spontaneous silent gestures may exhibit generalizable forms for many concepts across individuals (Ortega & Özyürek, 2016(Ortega & Özyürek, , 2019van Nispen et al, 2014). Importantly, some of these gestures overlap in form in varying degrees with conventionalised signs.…”

“…Relevant to this study is the finding that individuals often resort to similar iconic strategies to represent the same concepts in silent gesture (Ortega & Özyürek, 2016(Ortega & Özyürek, , 2019Van Nispen et al, 2017). Ortega and Özyürek (2019) created a database of elicited silent gestures from Dutch adults and they found that participants employ remarkably similar strategies to depict referents. Through the implementation of specific types of iconic depictions and their combinations, participants systematically represent concepts across different semantic domains.…”

“…For instance, it has been argued that in some sign languages, the acting strategy is commonly used to refer to actions (verbs) whereas the representing strategy is used to refer to objects (nouns) (Padden et al, 2015(Padden et al, , 2013. While this differentiations has not been observed in silent gesture (e.g., Ortega & Özyürek, 2019;Van Nispen, Van De Sandt-Koenderman, & Krahmer, 2017) the similarities in how gesture and sign represent concepts iconically support claims that both forms of manual communication share a common basis (Kendon, 2004(Kendon, , 2008. This common ground could give hearing nonsigners some leverage to recognise signs never seen before through their gestural knowledge.…”

“…Iconicity and the extent to which sign-naïve adults can access the meaning of iconic signs has been a central focus of attention in research. Based on evidence that many iconic gestures are highly systematic across individuals (Ortega & Özyürek, 2016(Ortega & Özyürek, , 2019Padden et al, 2015Padden et al, , 2013van Nispen, van de Sandt-Koenderman, Mol, & Krahmer, 2014) it is possible to assume that hearing non-signers share implicit knowledge of gestures for some concepts with specific forms and meanings on which they may base their judgment about signs.…”

Iconic gestures serve as manual cognates in hearing second language learners of a sign language: An ERP study.

“…The stimuli selection consisted of a two-stage procedure that involved i) collecting a set of iconic gestures that could be generalized across Dutch participants (silent gesture task). These gestures are a subset of a published database of 109 silent gestures (Ortega & Özyürek, 2019). Having collected these gestures, it was possible to ii) carry out a comparison between the form of each systematic gesture with its NGT sign equivalent (gesture-sign cross-comparison).…”