An integrated multi-level feature fusion framework for crowd behaviour prediction and analysis

Murthygowda, Manu Yadakere; Krishnegowda, Ravikumar Guralamata; Shashikala, S V

doi:10.11591/ijeecs.v30.i3.pp1369-1380

Cited by 6 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One common strategy is feature-level fusion, where features from different sensors are extracted and combined to form a comprehensive representation [13,14]. This approach often utilizes traditional feature extraction algorithms such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to extract useful features from image, audio, and textual data.…”

Section: Related Work 21 Multimodal Fusionmentioning

confidence: 99%

Multi-Modal Representation via Contrastive Learning with Attention Bottleneck Fusion and Attentive Statistics Features

Guo,

Liao,

et al. 2023

Entropy

View full text Add to dashboard Cite

The integration of information from multiple modalities is a highly active area of research. Previous techniques have predominantly focused on fusing shallow features or high-level representations generated by deep unimodal networks, which only capture a subset of the hierarchical relationships across modalities. However, previous methods are often limited to exploiting the fine-grained statistical features inherent in multimodal data. This paper proposes an approach that densely integrates representations by computing image features’ means and standard deviations. The global statistics of features afford a holistic perspective, capturing the overarching distribution and trends inherent in the data, thereby facilitating enhanced comprehension and characterization of multimodal data. We also leverage a Transformer-based fusion encoder to effectively capture global variations in multimodal features. To further enhance the learning process, we incorporate a contrastive loss function that encourages the discovery of shared information across different modalities. To validate the effectiveness of our approach, we conduct experiments on three widely used multimodal sentiment analysis datasets. The results demonstrate the efficacy of our proposed method, achieving significant performance improvements compared to existing approaches.

show abstract

Section: Related Work 21 Multimodal Fusionmentioning

confidence: 99%