Co-linguistic content inferences: From gestures to sound effects and emoji

Abstract

Among other uses, co-speech gestures can contribute additional semantic content to the spoken utterances with which they coincide. A growing body of research is dedicated to understanding how inferences from gestures interact with logical operators in speech, including negation ("not"/"n’t"), modals (e.g., "might"), and quantifiers (e.g., "each", "none", "exactly one"). A related but less-addressed question is what kinds of meaningful content other than gestures can evince this same behavior; this is in turn connected to the much broader question of what properties of gestures are responsible for how they interact with logical operators. We present two experiments investigating sentences with co-speech sound effects and co-text emoji in lieu of gestures, revealing a remarkably similar inference pattern to that of co-speech gestures. The results suggest that gestural inferences do not behave the way they do because of any traits specific to gestures, and that the inference pattern extends to a much broader range of content.

Semantic differences in visually similar face emojis

“…This is one important reason why emojis, which can take over many of the functions of such body movements, have become so popular. They have been argued to fulfill the role of gestures (McCulloch & Gawne, 2018;Pasternak & Tieu, 2022;Pierini, 2021) facial expressions (e.g., Weiß et al, 2020), particles (Beisswenger & Pappert, 2019), or discourse connectives (Beisswenger & Pappert, 2019) and to facilitate the understanding of indirect meaning (Holtgraves & Robinson, 2020).…”

Pragmatic Functions of Emoji on Chinese Social Media

“…Emojis are creative, convenient, and efficient substitutes for conveying non-textual information in an online setting. [7,8] Context is the key to analyze and interpret emojis [9]. The meaning of an emoji relies on the accompanied text if it exists.…”

“…It shapes the shared knowledge of people and determines how individuals perceive each visual element. [1,8] In addition, online platforms with different renderings of emojis can also affect individual's interpretation. [11] Thus, Bilibili and Weibo, which are both Chinese social media and have similar emoji renderings are chosen for this study.…”