Multimodal Analysis and Prediction of Latent User Dimensions

Wendlandt, Laura; Mihalcea, Rada; Boyd, Ryan L.; Pennebaker, James W.

doi:10.1007/978-3-319-67217-5_20

Cited by 11 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In general, using both text and image is helpful, a fact that is unsurprising since combinations of text and image are known to increase performance on tasks such as predicting post popularity or user personality (Hessel et al, 2017;Wendlandt et al, 2017). Most telling, however, were the differences in this helpfulness across items.…”

Section: Discussionmentioning

confidence: 99%

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts

Kruk

Lubin²,

Sikka

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Computing author intent from multimodal data like Instagram posts requires modeling a complex relationship between text and image. For example, a caption might evoke an ironic contrast with the image, so neither caption nor image is a mere transcript of the other. Instead they combine-via what has been called meaning multiplication Bateman (2014)-to create a new meaning that has a more complex relation to the literal meanings of text and image. Here we introduce a multimodal dataset of 1299 Instagram posts labeled for three orthogonal taxonomies: the authorial intent behind the image-caption pair, the contextual relationship between the literal meanings of the image and caption, and the semiotic relationship between the signified meanings of the image and caption. We build a baseline deep multimodal classifier to validate the taxonomy, showing that employing both text and image improves intent detection by 9.6% compared to using only the image modality, demonstrating the commonality of nonintersective meaning multiplication. The gain with multimodality is greatest when the image and caption diverge semiotically. Our dataset offers a new resource for the study of the rich meanings that result from pairing text and image.

show abstract

Section: Discussionmentioning

confidence: 99%

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts

Kruk

Lubin²,

Sikka

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…Например, в работе Л. Вендлэнд с соавт. показано, что совместное использование текстов и изображений заметно улучшало точность прогноза личностных черт «Большой пятерки» по сравнению с обособленным применением указанных цифровых следов (Wendlandt et al, 2017). Метаанализ, проведенный Д. Азукаром с соавт.…”

Section: Personal and Situational Factors Of Decision-making Under Trunclassified

Untitled

2020

psychology

View full text Add to dashboard Cite

Учре ди тель Национальный исследовательский уни вер си тет «Вы сшая шко ла эко но ми ки» Главный редактор В.А. Петровский (НИУ ВШЭ) Редакционная коллегия Дж. Берри (Университет Куинс, Кана да) Г.М. Бреслав (Балтийская международная академия, Латвия) Я. Вальсинер (Ольборгский университет, Дания) Е.Л. Григоренко (МГУ им. М.В. Ло мо но сова и Центр ребенка Йельского университета, США) В.А. Ключарев (НИУ ВШЭ) Д.А. Леонтьев (НИУ ВШЭ и МГУ им. М.В. Ломоносова) В.А. Лефевр (Калифорнийский университет, США) М. Линч (Рочестерский университет, США) Д.В. Люсин (НИУ ВШЭ и ИП РАН) Е.Н. Осин (НИУ ВШЭ) А.Н. Поддьяков (НИУ ВШЭ) Е.Б. Старовойтенко (НИУ ВШЭ) Д.В. Ушаков (зам. глав. ред.) (ИП РАН) М.В. Фаликман (НИУ ВШЭ) А.В. Хархурин (НИУ ВШЭ) В.Д. Шадриков (зам. глав. ред.) (НИУ ВШЭ) С.Р. Яголковский (зам. глав. ред.) (НИУ ВШЭ) Экспертный совет К.А. Абульханова-Славская (НИУ ВШЭ и ИП РАН) Н.А. Алмаев (ИП РАН) В.А. Барабанщиков (ИП РАН и МГППУ) Т.Ю. Базаров (НИУ ВШЭ и МГУ им. М.В. Ломоносова) А.К. Болотова (НИУ ВШЭ) А.Н. Гусев (МГУ им. М.В. Ломоносова) А.Л. Журавлев (ИП РАН) А.В. Карпов (Ярославский государст венный университет им. П.Г. Демидова) П. Лучисано (Римский университет Ла Сапиенца, Италия) А. Лэнгле (НИУ ВШЭ) А.Б. Орлов (НИУ ВШЭ) В.Ф. Петренко (МГУ им. М.В. Ломо но сова) В.М. Розин (ИФ РАН) И.H. Семенов (НИУ ВШЭ) Е.А. Сергиенко (ИП РАН) Т.Н. Ушакова (ИП РАН) А.М. Черноризов (МГУ им. М.В. Ломо но со ва) А.Г. Шмелев (МГУ им. М.В. Ломо но сова) П. Шмидт (НИУ ВШЭ и Гиссенский университет, Германия)

show abstract

“…In (Wendlandt et al, 2017), the authors use a corpus with images and text from students. The work was about classified gender and personality.…”

Section: Multimodal Based Approachesmentioning

confidence: 99%

“…Nevertheless, one of the most popular types of information for the AP task is the images that the users share. Some authors have used this information obtaining good results (You et al, 2014;Schwartz et al, 2013a;Wendlandt et al, 2017).…”

Section: Albert Einsteinmentioning

confidence: 99%

Author Profiling in Social Media with Multimodal Information

Carmona¹,

Tello²,

Gómez³

et al. 2020

CyS

View full text Add to dashboard Cite

Determine aspects of a person as gender, age, residency, occupation, among others, through his/her texts is a task that is part of the natural language processing and is known as author profiling.In this thesis work, we propose a solution for the task of profiling authors in social networks. Our solution uses a multimodal approach to extracting information from Agradecimientos Mamá, no hay palabras para poder agradecerle todo tu apoyo a lo largo de toda una vida y más. ¡Simplemente Gracias! Mi agradecimiento al pueblo mexicano y al Consejo Nacional de Ciencia y Tecnología (CONACYT) por el apoyo otorgado a través de la beca no. 401887. También agradezco el apoyo que se dio a esta tesis a través de los proyectos CB-2015-01-258588,

show abstract

Multimodal Analysis and Prediction of Latent User Dimensions

Cited by 11 publications

References 36 publications

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts

Untitled

Author Profiling in Social Media with Multimodal Information

Contact Info

Product

Resources

About