Multimodal research is an emerging field of artificial intelligence, and one of the main research problems in this field is multimodal fusion. The fusion of multimodal data is the process of integrating multiple unimodal representations into one compact multimodal representation. Previous research in this field has exploited the expressiveness of tensors for multimodal representation. However, these methods often suffer from exponential increase in dimensions and in computational complexity introduced by transformation of input into tensor. In this paper, we propose the Lowrank Multimodal Fusion method, which performs multimodal fusion using low-rank tensors to improve efficiency. We evaluate our model on three different tasks: multimodal sentiment analysis, speaker trait analysis, and emotion recognition. Our model achieves competitive results on all these tasks while drastically reducing computational complexity. Additional experiments also show that our model can perform robustly for a wide range of low-rank settings, and is indeed much more efficient in both training and inference compared to other methods that utilize tensor representations.

Humans convey their intentions through the usage of both verbal and nonverbal behaviors during face-to-face communication. Speaker intentions often vary dynamically depending on different nonverbal contexts, such as vocal patterns and facial expressions. As a result, when modeling human language, it is essential to not only consider the literal meaning of the words but also the nonverbal contexts in which these words appear. To better model human language, we first model expressive nonverbal representations by analyzing the fine-grained visual and acoustic patterns that occur during word segments. In addition, we seek to capture the dynamic nature of nonverbal intents by shifting word representations based on the accompanying nonverbal behaviors. To this end, we propose the Recurrent Attended Variation Embedding Network (RAVEN) that models the fine-grained structure of nonverbal subword sequences and dynamically shifts word representations based on nonverbal cues. Our proposed model achieves competitive performance on two publicly available datasets for multimodal sentiment analysis and emotion recognition. We also visualize the shifted word representations in different nonverbal contexts and summarize common patterns regarding multimodal variations of word representations. Negative-shifted word representation Original word representation Positive-shifted word representation Visual Acoustic ⋯ excited voice raised eyebrows Visual Acoustic ⋯ soft voice shock Word Representation Space

Unsafe acts contribute dominantly to construction accidents, and increasing safety behavior is essential to reduce accidents. Previous research conceptualized safety behavior as an interaction between proximal individual differences (safety knowledge and safety motivation) and distal contextual factors (leadership and safety climate). However, relatively little empirical research has examined this conceptualization in the construction sector. Given the cultural background of the sample, this study makes a slight modification to the conceptualization and views transformational leadership as an antecedent of safety climate. Accordingly, this study establishes a multiple mediator model showing the mechanisms through which transformational leadership translates into safety behavior. The multiple mediator model is estimated by the structural equation modeling (SEM) technique, using individual questionnaire responses from a random sample of construction personnel based in Hong Kong. As hypothesized, transformational leadership has a significant impact on safety climate which is mediated by safety-specific leader–member exchange (LMX), and safety climate in turn impacts safety behavior through safety knowledge. The results suggest that future safety climate interventions should be more effective if supervisors exhibit transformational leadership, encourage construction personnel to voice safety concerns without fear of retaliation, and repeatedly remind them about safety on the job.

No abstract