Emoji-based semantic representations for abstract and concrete concepts

Abstract

An increasingly large body of converging evidence supports the idea that the semantic system is distributed across brain areas and that the information encoded therein is multimodal. Within this framework, feature norms are typically used to operationalize the various parts of meaning that contribute to define the distributed nature of conceptual representations. However, such features are typically collected as verbal strings, elicited from participants in experimental settings. If the semantic system is not only distributed (across features) but also multimodal, a cognitively sound theory of semantic representations should take into account different modalities in which feature-based representations are generated, because not all the relevant semantic information may be easily verbalized into classic feature norms, and different types of concepts (e.g., abstract vs. concrete concepts) may consist of different configurations of non-verbal features.In this paper we acknowledge the multimodal nature of conceptual representations and we propose a novel way of collecting non-verbal semantic features. In a crowdsourcing task we asked participants to use emoji to provide semantic representations for a sample of 300 English nouns referring to abstract and concrete concepts, which account for (machine readable) visual features. In a formal content analysis with multiple annotators we then classified the cognitive strategies used by the participants to represent conceptual content through emoji.The main results of our analyses show that abstract (vs. concrete) concepts are characterized by representations that: 1. consist of a larger number of emoji; 2. include more face emoji (expressing emotions); 3. are less stable and less shared among users; 4. use representation strategies based on figurative operations (e.g., metaphors) and strategies that exploit linguistic information (e.g. rebus); 5. correlate less well with the semantic representations emerging from classic features listed through verbal strings.

The Multidimensional Lexicon of Emojis: A New Tool to Assess the Emotional Content of Emojis

“…On a practical level, we anticipated that participants would have a difficult time producing reliable and useful emotion ratings for many of these emojis provided in isolation (e.g., the degree of sadness represented by a single piece of fruit or baseball), and they were not developed with this purpose in mind. Indeed, recent research has shown that face emojis are more frequently used to represent abstract concepts, compared to concrete concepts (Wicke and Bolognesi, 2020).…”

“…Our study was also limited by only including emojis from the Smileys and Emotion category in the human raters analysis of Study 2. We made this choice due to the potential for emoji users to have difficulty identifying the emotional content of non-face emojis when viewed out of context, despite their importance in emotional communication (Riordan, 2017b), use in representing abstract and concrete concepts (Wicke and Bolognesi, 2020), and inclusion in other emoji lexicons (Novak et al, 2015). At the same time, future research could use more complex methods to obtain human ratings for the emotional content of non-face emojis, such as asking raters to match emojis to posts or messages with known emotional valences.…”

A multi-modal and multi-scale emotion-enhanced inference model based on fuzzy recognition

“…The features of given texts and emojis could be automatically extracted by a multi-modal Siamese-based framework [38]. To represent the conceptual content through emoji from cognitive strategies, participants were asked to use emoji to provide semantic representations for abstract and concrete concepts [39].…”

Semantic Journeys: Quantifying Change in Emoji Meaning from 2012-2018

“…More specifically, Naumann, Frassinelli, and Schulte im Walde (2018) found partial support for the hypothesis that concrete words have significantly less diverse contexts compared to abstract words. Given their specific characteristics, emoji represent an important linguistic category to investigate alongside words with respect to properties along the concrete/abstract scale (Wicke and Bolognesi 2020). To investigate the relation between concrete versus abstract emoji and their degree of semantic change, we devised a pipeline drawing on EmojiNet (Wijeratne et al 2017), BabelNet (Navigli and Ponzetto 2012) and the concreteness dataset by Brysbaert, Warriner, and Kuperman (2014).…”