DANKMEMES @ EVALITA 2020: The Memeing of Life: Memes, Multimodality and Politics

Miliani, Martina; Giorgi, Giulia; Rama, Ilir; Anselmi, Guido; Lebani, Gianluca E.

doi:10.4000/books.aaccademia.7330

Cited by 15 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…automatically discriminating image memes from regular images, has not yet received considerable attention from the research community. To our knowledge, there is only one dataset for meme detection, namely the DankMemes dataset [17], which was released in 2020 but is publicly unavailable at the timing of writing the paper. DankMemes contains 2000 images related to the 2019 Italian government crisis, half of which are memes and the rest regular images.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

MemeTector: Enforcing deep focus for meme detection

Koutlis¹,

Schinas²,

Papadopoulos³

2022

Preprint

View full text Add to dashboard Cite

Image memes and specifically their widely-known variation image macros, is a special new media type that combines text with images and is used in social media to playfully or subtly express humour, irony, sarcasm and even hate. It is important to accurately retrieve image memes from social media to better capture the cultural and social aspects of online phenomena and detect potential issues (hate-speech, disinformation). Essentially, the background image of an image macro is a regular image easily recognized as such by humans but cumbersome for the machine to do so due to feature map similarity with the complete image macro. Hence, accumulating suitable feature maps in such cases can lead to deep understanding of the notion of image memes. To this end, we propose a methodology that utilizes the visual part of image memes as instances of the regular image class and the initial image memes as instances of the image meme class to force the model to concentrate on the critical parts that characterize an image meme. Additionally, we employ a trainable attention mechanism on top of a standard ViT architecture to enhance the model's ability to focus on these critical parts and make the predictions interpretable. Several training and test scenarios involving web-scraped regular images of controlled text presence are considered in terms of model robustness and accuracy. The findings indicate that light visual part utilization combined with sufficient text presence during training provides the best and most robust model, surpassing state of the art.

show abstract

Section: Related Workmentioning

confidence: 99%

“…The latter works utilize datasets with image memes and appropriate labeling, thus they do not put effort on detecting the memes to be analysed. The detection of image memes and their discrimination from regular images is currently still a relatively understudied topic, and there are only few attempts in this direction [17,18].…”

Section: Introductionmentioning

confidence: 99%

MemeTector: Enforcing deep focus for meme detection

Koutlis¹,

Schinas²,

Papadopoulos³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…One similar work to ours is sticker recommendation (Jesus et al, 2019;Laddha et al, 2020;Gao et al, 2020), where suitable stickers are retrieved to match the text-only dialogue history, which can be regarded as a special case of MOD. Besides, the tasks of meme retrieval (Milo et al, 2019;Perez-Martin et al, 2020;Sharma et al, 2020), detecting the hate speech in memes and clustering memes according to events (Miliani et al, 2020;Kiela et al, 2020) are proposed to help the Internet meme modeling. In this work, we focus on a more challenging situation, generating Internet meme merged utterances to make conversations more vivid and engaging.…”

Section: Internet Meme In Dialoguementioning

confidence: 99%

Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark

Fei¹,

Li²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

As a kind of new expression elements, Internet memes are popular and extensively used in online chatting scenarios since they manage to make dialogues vivid, moving, and interesting. However, most current dialogue researches focus on text-only dialogue tasks. In this paper, we propose a new task named as Meme incorporated Open-domain Dialogue (MOD). Compared to previous dialogue tasks, MOD is much more challenging since it requires the model to understand the multimodal elements as well as the emotions behind them. To facilitate the MOD research, we construct a large-scale open-domain multimodal dialogue dataset incorporating abundant Internet memes into utterances. The dataset consists of ∼45K Chinese conversations with ∼606K utterances. Each conversation contains about 13 utterances with about 4 Internet memes on average and each utterance equipped with an Internet meme is annotated with the corresponding emotion. In addition, we present a simple and effective method, which utilizes a unified generation network to solve the MOD task. Experimental results demonstrate that our method trained on the proposed corpus is able to achieve expressive communication including texts and memes. The corpus and models have been publicly available at https:// github.com/lizekang/DSTC10-MOD.

show abstract

“…The macro F1-score for the baseline model was 0.21 for sentiment and 0.50 for humor type classification using image-text models, emphasizing how challenging interpreting memes can be. Additionally [15] is a similar dataset, but with 2,631 Italian memes.…”

Section: Related Work 31 Related Datasetsmentioning

confidence: 99%

An Interpretable Approach to Hateful Meme Detection

Deshpande¹,

Mani

2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Hateful memes are an emerging method of spreading hate on the internet, relying on both images and text to convey a hateful message. We take an interpretable approach to hateful meme detection, using machine learning and simple heuristics to identify the features most important to classifying a meme as hateful. In the process, we build a gradient-boosted decision tree and an LSTM-based model that achieve comparable performance (73.8 validation and 72.7 test auROC) to the gold standard of humans and state-of-the-art transformer models on this challenging task. CCS CONCEPTS• Computing methodologies → Artificial intelligence; Machine learning.

show abstract

DANKMEMES @ EVALITA 2020: The Memeing of Life: Memes, Multimodality and Politics

Cited by 15 publications

References 28 publications

MemeTector: Enforcing deep focus for meme detection

MemeTector: Enforcing deep focus for meme detection

Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark

An Interpretable Approach to Hateful Meme Detection

Contact Info

Product

Resources

About