The context, such as scenes and objects, plays an important role in video emotion recognition. The emotion recognition accuracy can be further improved when the context information is incorporated. Although previous research has considered the context information, the emotional clues contained in different images may be different, which is often ignored. To address the problem of emotion difference between different modes and different images, this paper proposes a hierarchical attention-based multimodal fusion network for video emotion recognition, which consists of a multimodal feature extraction module and a multimodal feature fusion module. The multimodal feature extraction module has three subnetworks used to extract features of facial, scene, and global images. Each subnetwork consists of two branches, where the first branch extracts the features of different modes, and the other branch generates the emotion score for each image. Features and emotion scores of all images in a modal are aggregated to generate the emotion feature of the modal. The other module takes multimodal features as input and generates the emotion score for each modal. Finally, features and emotion scores of multiple modes are aggregated, and the final emotion representation of the video will be produced. Experimental results show that our proposed method is effective on the emotion recognition dataset.

No abstract

No abstract

No abstract

The inversion of directions is an important operation with directions which plays an important role in qualitative spatial reasoning and spatial queries. In this work, we address on the inversion operation of the basic cardinal direction relations in the model of Goyal. The direction relation matrix model proposed by Goyal is a projection-based model for spatial direction relations between regions. This model is simple in calculation and easy to carry out formal reasoning, which is considered as currently one of the most excellent models for representation and qualitative reasoning with cardinal direction relations in two-dimensional space. This work aims to realize the automatic inference and calculation of the inverse of the basic cardinal direction relations in the model of Goyal and further to improve the ability of spatial reasoning and spatial analysis of spatial database. In order to avoid the complicated manual reasoning, an algorithm for automatically performing the inverse operation on this model is devised by means of the operations of direction relation matrix. Theorems are provided to prove formally that our algorithm is correct and complete, which is also verified by comparing the result of our algorithm with that of manual reasoning for each basic cardinal direction relation. This study realized the automatic inference and calculation of the inverse of the basic cardinal direction relations in the model of Goyal and further improved the ability of spatial reasoning and spatial analysis of spatial database.