M-SENA: An Integrated Platform for Multimodal Sentiment Analysis

Huisheng, Mao,; Yuan, Zhiquan; Xu, Hui; Yu, Wenmeng; Yihe, Liu,; Gao, Kai

doi:10.48550/arxiv.2203.12441

Cited by 3 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, a pretrained Wav2vec2 model (Baevski et al 2020) is used to generate timestamps for video/audio to text alignment. All above customized feature extraction are performed with the help of MMSA-FET toolkit (Mao et al 2022). Integrated MSA Models.…”

Section: Noise Defence Methodsmentioning

confidence: 99%

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Huisheng

Zhang

et al. 2023

AAAI

View full text Add to dashboard Cite

Improving model robustness against potential modality noise, as an essential step for adapting multimodal models to real-world applications, has received increasing attention among researchers. For Multimodal Sentiment Analysis (MSA), there is also a debate on whether multimodal models are more effective against noisy features than unimodal ones. Stressing on intuitive illustration and in-depth analysis of these concerns, we present Robust-MSA, an interactive platform that visualizes the impact of modality noise as well as simple defence methods to help researchers know better about how their models perform with imperfect real-world data.

show abstract

Section: Noise Defence Methodsmentioning

confidence: 99%

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Huisheng

Zhang

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…For traditional neural networks, we adopt consistent parameter settings with the approach outlined in prior work [33] and evaluate them on Mixed Set, Conflicting Set and Aligned Set. All experiments are conducted on one NVIDIA A10 GPU.…”

Section: Methodsmentioning

confidence: 99%

Learning ability of top university students in China: Shanghai Jiao Tong University as a case study

Wang

2022

SN Soc Sci

View full text Add to dashboard Cite

The network illustrates the interrelationships among diverse entities and has attracted considerable attention. A wide range of applications makes the network a popular modeling tool, including but not limited to social networks [1], business administration [2], and city construction [3]. In these fields, quantifying network value is essential. By examining network value, companies gain insights into consumer behavior, allowing them to refine marketing strategies accordingly [4]. Similarly, investigating network value can assist urban planners in better understanding the utilization and demands for public facilities, ultimately optimizing city construction [5]. To better understand network value, researchers have established some influential laws that describe network value in terms of the number of neighbors, edges, and subgraphs.Sarnoff 's Law. In the 1940s, David Sarnoff proposed that the value of a broadcasting network is directly proportional to the number of nodes (audience), which is also known as Sarnoff's law [6]. For example, in the case of TV programs, the network value increases linearly with the number of the audience, because the growing number of audiences allows advertisers to access more potential customers. This results in more advertisers, increased revenues, and higher broadcast media demand. Formally, for a network with n nodes, the communication value is Θ(n). This law was initially applied to the film industry, later extended to television, and usually represents one-way communication. Broadcasting can only transmit messages unidirectionally to users, but cannot spread information within users.Metcalfe's Law. With the conferral of the Turing Award on Robert Metcalfe in 2022, Metcalfe's law[7] has regained attention. Metcalfe's law was proposed on the background of the increasing number of Ethernet users and growing attention to the interconnection value of networks. Metcalfe argues that if the value of each node (terminal) is equal, the value of a network is proportional to the number of edges. One of the classic illustrations of Metcalfe's law lies in communication networks, where a network with n users can provide interconnection value proportional to approximately n 2 , i.e. Θ(n 2 ), as each user can communicate with the other n − 1 users in the network. Despite the many challenges associated with the development of the network, such as network scale, connection quality, and network design, Metcalfe's law remains applicable in some scenarios, especially in the field of cloud computing [8]. The increasing number of cloud computing users leads to the availability of more resources, which in turn attracts additional users, creating a virtuous cycle of growth. As a theory for describing the interconnection value of networks, Metcalfe's law is held in high regard and possesses significant implications for communication networks, the Internet, and social networks.Reed's Law. In 2000, David Reed proposed Reed's law [9] for group-forming networks (GFNs). He argued that network value depends not ...

show abstract

“…It utilizes images as attention anchors to emphasize key sentences in the text. The M-SENA platform stands as a significant contribution to advanced multimodal sentiment analysis, offering not only an open-sourced framework with flexible toolkits and reliable benchmarks but also a modular video SA architecture, contributing valuable resources to the research community [23]. In [24], a pioneering framework known as MWRCMH fusion is presented, addressing the challenges inherent in real-world multimodal sentiment analysis.…”

Section: Research On Multimodal Sentiment Analysismentioning

confidence: 99%

Design and Efficacy of a Data Lake Architecture for Multimodal Emotion Feature Extraction in Social Media

Fan,

2024

IET Software

View full text Add to dashboard Cite

In the rapidly evolving landscape of social media, the demand for precise sentiment analysis (SA) on multimodal data has become increasingly pivotal. This paper introduces a sophisticated data lake architecture tailored for efficient multimodal emotion feature extraction, addressing the challenges posed by diverse data types. The proposed framework encompasses a robust storage solution and an innovative SA model, multilevel spatial attention fusion (MLSAF), adept at handling text and visual data concurrently. The data lake architecture comprises five layers, facilitating real-time and offline data collection, storage, processing, standardized interface services, and data mining analysis. The MLSAF model, integrated into the data lake architecture, utilizes a novel approach to SA. It employs a text-guided spatial attention mechanism, fusing textual and visual features to discern subtle emotional interplays. The model’s end-to-end learning approach and attention modules contribute to its efficacy in capturing nuanced sentiment expressions. Empirical evaluations on established multimodal sentiment datasets, MVSA-Single and MVSA-Multi, validate the proposed methodology’s effectiveness. Comparative analyses with state-of-the-art models showcase the superior performance of our approach, with an accuracy improvement of 6% on MVSA-Single and 1.6% on MVSA-Multi. This research significantly contributes to optimizing SA in social media data by offering a versatile and potent framework for data management and analysis. The integration of MLSAF with a scalable data lake architecture presents a strategic innovation poised to navigate the evolving complexities of social media data analytics.

show abstract

M-SENA: An Integrated Platform for Multimodal Sentiment Analysis

Cited by 3 publications

References 22 publications

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Learning ability of top university students in China: Shanghai Jiao Tong University as a case study

Design and Efficacy of a Data Lake Architecture for Multimodal Emotion Feature Extraction in Social Media

Contact Info

Product

Resources

About